Capturing changes in video frames was the basis of an earlier post, https://www.cloudacm.com/?p=3423. The intent was to trim out the content that had little or no changes between frames. This saved storage space so more content could be retained. I had stumbled upon this demonstration that took a different approach.

This method puts emphasis on the changes between frames by highlighting the differences. The areas with no change becomes the background. It reminded me of an earlier demo that I had seen years before that used FFMPEG.

The process basically takes two different frames in a video and processes the changes in pixels. I had also seen demos of motion amplification, but that topic will not be covered here.

This sample video was processed with FFMPEG based off of Dave Rice’s blog post, https://dericed.com/2012/display-video-difference-with-ffmpegs-overlay-filter/. The script used is shown below the video.

#!/bin/bash
# FFMPEG Native Motion Extraction Process
# Source - https://dericed.com/2012/display-video-difference-with-ffmpegs-overlay-filter/

ffmpeg -i 20220324_085905_01.MOV -ss 4 -to $(ffmpeg -i 20220324_085905_01.MOV 2>&1 | grep "Duration" | awk '{print $2}' | tr -d , | awk -F: '{ print ($1 * 3600) + ($2 * 60) + $3 - 4 }') -c copy 20220324_085905_01_TrimLead.MOV
sleep 3
ffmpeg -y -i 20220324_085905_01_TrimLead.MOV -i 20220324_085905_01.MOV -filter_complex '[1:v]format=yuva444p,lut=c3=128,negate[video2withAlpha],[0:v][video2withAlpha]overlay[out]' -map [out] 20220324_085905_01_Output.MOV
sleep 3
ffmpeg -i 20220324_085905_01.MOV -i 20220324_085905_01_Output.MOV -filter_complex "hstack,format=yuv420p" -c:v libx264 -crf 18 20220324_085905_01_SideBySide.MOV
sleep 3
ffmpeg -i 20220324_085905_01_SideBySide.MOV -vf scale="960:270" MotionExtractionDemo-1.MOV
exit

The following video is a modified FFMPEG process that was suggested on the Stackoverflow forum, https://stackoverflow.com/questions/25774996/how-to-compare-show-the-difference-between-2-videos-in-ffmpeg. The script for it is provided below the video.

#!/bin/bash
# FFMPEG Native Motion Extraction Process
# Source - https://stackoverflow.com/questions/25774996/how-to-compare-show-the-difference-between-2-videos-in-ffmpeg

ffmpeg -i 20220324_085905_01.MOV -ss 4 -to $(ffmpeg -i 20220324_085905_01.MOV 2>&1 | grep "Duration" | awk '{print $2}' | tr -d , | awk -F: '{ print ($1 * 3600) + ($2 * 60) + $3 - 4 }') -c copy 20220324_085905_01_TrimLead.MOV
sleep 3
ffmpeg -i 20220324_085905_01_TrimLead.MOV -i 20220324_085905_01.MOV -filter_complex "blend=all_mode=difference" -c:v libx264 -crf 18 -c:a copy 20220324_085905_01_Output.MOV
sleep 3
ffmpeg -i 20220324_085905_01.MOV -i 20220324_085905_01_Output.MOV -filter_complex "hstack,format=yuv420p" -c:v libx264 -crf 18 20220324_085905_01_SideBySide.MOV
sleep 3
ffmpeg -i 20220324_085905_01_SideBySide.MOV -vf scale="960:270" MotionExtractionDemo-2.MOV
exit

This following video is a modification of previous which was provided from a prompt to OpenAI’s ChatGPT. The intent was to make the background black with highlights in white. The script follows the video below.

#!/bin/bash
# FFMPEG Native Motion Extraction Process

ffmpeg -i 20220324_085905_01.MOV -ss 4 -to $(ffmpeg -i 20220324_085905_01.MOV 2>&1 | grep "Duration" | awk '{print $2}' | tr -d , | awk -F: '{ print ($1 * 3600) + ($2 * 60) + $3 - 4 }') -c copy 20220324_085905_01_TrimLead.MOV
sleep 3
ffmpeg -i 20220324_085905_01_TrimLead.MOV -i 20220324_085905_01.MOV -filter_complex "blend=all_mode=difference,format=gray,lut=y=val*2" -c:v libx264 -crf 18 -c:a copy 20220324_085905_01_Output.MOV
sleep 3
ffmpeg -i 20220324_085905_01.MOV -i 20220324_085905_01_Output.MOV -filter_complex "hstack,format=yuv420p" -c:v libx264 -crf 18 20220324_085905_01_SideBySide.MOV
sleep 3
ffmpeg -i 20220324_085905_01_SideBySide.MOV -vf scale="960:270" MotionExtractionDemo-3.MOV
exit

This last demo is a process that uses ImageMagick’s compare command, https://usage.imagemagick.org/compare/. The process takes longer and isn’t as elegant as FFMPEG. However, the variety of adjustments to the results makes this something worth considering.

#!/bin/bash
# Motion Extraction Process with ImageMagick

# Set all the script variables
srcnme=20220324_085905_01
srctype=.MOV
wrkdir=/home/MotionExtraction/

srcdir=Images
srcimg=${wrkdir}${srcdir}/
srcmov=${srcnme}${srctype}
imgseq=${srcimg}${srcnme}_%04d.png

prcamnt3=4SecLead
prcimg3=${wrkdir}${prcamnt3}/
outputmov3=${srcnme}_${prcamnt3}${srctype}

factoramount3=6
framecount=300

cd $wrkdir

mkdir $srcdir
mkdir $prcamnt3

# Extract images from video frames
ffmpeg -i ${srcmov} ${imgseq}

# Process each image
cd $prcimg3
filecount=1

until [ $filecount -gt $((framecount - factoramount3)) ]
do
  filepadding=`printf "%04d" $filecount`
  ((leadfile=filecount+factoramount3))
  leadpadding=`printf "%04d" $leadfile`
  
  convert ${srcimg}${srcnme}_${filepadding}.png ${srcimg}${srcnme}_${leadpadding}.png -compose difference -composite ${prcimg3}${srcnme}_${prcamnt3}_${filepadding}.png
  ((filecount=filecount+1))
done

# Build video from image frames
ls *.png > imagelist1.txt
sed "s/${srcnme}/file \'${srcnme}/" imagelist1.txt > imagelist2.txt
sed "s/png/png\'/" imagelist2.txt > imagelist3.txt
ffmpeg -y -r 30 -f concat -safe 0 -i "imagelist3.txt" -c:v libx264 -vf "fps=30,format=yuv420p" ${outputmov3}
rm *.txt

# Create Side by Side comparison video
ffmpeg -i ${srcmov} -i ${outputmov3} -filter_complex "hstack,format=yuv420p" -c:v libx264 -crf 18 20220324_085905_01_SideBySide.MOV
sleep 3
ffmpeg -i 20220324_085905_01_SideBySide.MOV -vf scale="960:270" MotionExtractionDemo-4.MOV
exit