In this post, we’ll step through how to create anaglyphic stereoscopic videos using FFMpeg. To create anaglyph imaging, we’ll need to use two camera sources, one left, the other right. These sources will be merged into a single image with red cyan overlaying to create a three dimensional effect using red cyan glasses. I referenced these sites frequently.

FFMpeg Documentation – https://ffmpeg.org/documentation.html
FFMpeg Stereoscopic Details – https://trac.ffmpeg.org/wiki/Stereoscopic

The source video I’ll be demonstrating was shot with two Innovv C1 dashboard cameras that I was able to buy online for $50 each. They have since been replaced with newer generation models that offer more durability, quality, and a higher price tag. These cameras were attached to a steel bar and mounted on a tripod. The camera positioning was mostly an estimation, they were not mounted with measured precision. The footage is of riders participating in the annual Seattle to Portland Bicycle Classic event, shot in July of 2017. The video was taken in the University District, mostly on the University Bridge.

Start with getting a sample image from each of the cameras. This cuts down on processing time and will aid in the refinement.

Open the images with Gimp as layers and set them with an opacity of 50 percent. For the purposes of this demonstration, I’m going to export this to a new image and open it in Pinta. This new image has some problems. I’m going to draw lines between similar points to show what I mean. The red lines run at different angles. The ideal would be for them all to run perfectly horizontal. If I were to anaglyph these, the results would be visually straining.

To attempt to fix this, we’ll defish the lens distortion.

ffmpeg -i "Cam1-Left.png" -vf "lenscorrection=cx=0.5:cy=0.5:k1=-0.227:k2=-0.022" "Cam1-Left-DeFish.png"
ffmpeg -i "Cam2-Right.png" -vf "lenscorrection=cx=0.5:cy=0.5:k1=-0.227:k2=-0.022" "Cam2-Right-DeFish.png"

Still, there is some angular differences that need to be worked out. To correct this, we’ll rotate one of the images to get the alignment closer. I used Gimp with the Layer / Transform / Arbitrary Rotation command to trail and error the closest setting, which was 1.5 degrees. Now we’ll apply this to our right image.

ffmpeg -i "Cam2-Right-DeFish.png" -vf rotate=1.5*(PI/180) "Cam2-Right-DeFish-Rotate.png"

The angles follow a closer path, but they still aren’t the ideal horizontal we’ll want. So to fix this, we’ll move the right image up by a few pixels to get better alignment. Using Gimp, the adjustment to the right image was best when moved up 4 pixels. This is the best horizontal allignment.

However, the background objects should be closer, because they are the dimensional reference. To correct that we’ll move our right image over to the right by 42 pixels. This is much better. Now we’ll use a crop command in FFMpeg and apply this to both of our source images.

The original images are 1920×1080. For the right image we’ll trim 4 pixels off the top and 42 pixels from the left side. The final image will be 1878×1076

ffmpeg -i "Cam2-Right-DeFish-Rotate.png" -filter:v "crop=1878:1076:0:4" -c:a copy "Cam2-Right-DeFish-Rotate-Crop.png"

Our Left image also needs to be cropped. This time we’ll trim 4 pixels from the bottom and 42 pixels from the right. This will mimic the that 4×42 pixel move.

ffmpeg -i "Cam1-Left-DeFish.png" -filter:v "crop=1878:1076:42:0" -c:a copy "Cam1-Left-DeFish-Crop.png"

We’re almost there, now we need to trim out the dead space introduced with the rotation of our right image. So we’ll run another crop command on each of our images to remove 20 pixels from the outside edges of both images. The final image resolution will be 1838×1036

ffmpeg -i "Cam2-Right-DeFish-Rotate-Crop.png" -filter:v "crop=1838:1036:20:20" -c:a copy "Cam2-Right-DeFish-Rotate-Crop-Trim.png"
ffmpeg -i "Cam1-Left-DeFish-Crop.png" -filter:v "crop=1838:1036:20:20" -c:a copy "Cam1-Left-DeFish-Crop-Trim.png"

The previous trim and crop functions can be combines to save processing time. I’ll apply the combination on the final video process.

Now all of this could have be avoided if I had taken the time to align the hardware correctly. But what fun would been? This is a perfect example of how software can fix a hardware fault. Anyway now back to the intent of this post and the creation of an anaglyph.

First we’ll need to place both images side by side. Here is the command to do that.

ffmpeg -i "Cam2-Right-DeFish-Rotate-Crop-Trim.png" -i "Cam1-Left-DeFish-Crop-Trim.png" -filter_complex "[0:v]setpts=PTS-STARTPTS, pad=iw*2:ih[bg]; [1:v]setpts=PTS-STARTPTS[fg]; [bg][fg]overlay=w" "SideBySide.png"

SideBySide.png

The cross eyed check confirms. Now lets run our actual anaglyph command. We’re using ‘sbs2r’, side by side cross eye with half width resolution (right eye left, left eye right)

ffmpeg -i "SideBySide.png" -vf stereo3d=sbs2r:arch "SideBySide-Anaglyph.png"

SideBySide-Anaglyph.png

Looking through the anaglyph glasses validates the depth of objects. That was a lot of work to get the alignment corrected, but now we can apply these filters to the footage and create some anaglyph movies.

So here’s a recap of the commands we’ll run on the full length videos.

Remove Fisheye Distortion

ffmpeg -i "L0000004.MOV" -vf "lenscorrection=cx=0.5:cy=0.5:k1=-0.227:k2=-0.022" "L0000004-Left-Defish.MOV"
ffmpeg -i "L0000004.MOV" -vf "lenscorrection=cx=0.5:cy=0.5:k1=-0.227:k2=-0.022" "L0000004-Right-Defish.MOV"

Rotate the Right Camera

ffmpeg -i "L0000004-Right-Defish.MOV" -vf rotate=1.5*(PI/180) "L0000004-Right-Defish-Rotate.MOV"

Align and Trim both Cameras, done by adding the last x,y values together

ffmpeg -i "L0000004-Right-Defish-Rotate.MOV" -filter:v "crop=1838:1036:20:24" -c:a copy "L0000004-Right-Defish-Rotate-Trim.MOV"
ffmpeg -i "L0000004-Left-Defish.MOV" -filter:v "crop=1838:1036:62:20" -c:a copy "L0000004-Left-Defish-Trim.MOV"

Create Side by Side

ffmpeg -i "L0000004-Right-Defish-Rotate-Trim.MOV" -i "L0000004-Left-Defish-Trim.MOV" -filter_complex "[0:v]setpts=PTS-STARTPTS, pad=iw*2:ih[bg]; [1:v]setpts=PTS-STARTPTS[fg]; [bg][fg]overlay=w" "L0000004-SideBySide.MOV"

Finalize with the anaglyph

ffmpeg -i "L0000004-SideBySide.MOV" -vf stereo3d=sbs2r:arch "L0000004-SideBySide-Anaglyph.MOV"

This post drew from many of the topics I covered in earlier posts. It brings together the facets of FFMpeg processing to create a final result that is remarkable. It has been a rewarding effort to unfold what FFMpeg can do. I hope you have found this useful and that you can join me as we look further into what FFMpeg can do.