In this post, we’ll cover how to use FFMpeg to removed the radial effects of a fish eye lens. This distortion is a trade off of wide angle lenses. The wider view also included bends and bows to objects that are in nature straight. FFMpeg has a filter known as “lenscorrection” that will remove most of this distortion, lets begin.

As always, split out a segment so you don’t waste time processing excessive amounts of data.

ffmpeg -ss 00:00:39 -i /home/local/Desktop/MINI0016.MOV -t 00:00:02 -c copy /home/local/Desktop/DeFish/MINI0016_DeFish.MOV

Let’s take it a step further and split out the video into a series of images.

ffmpeg -i /home/local/Desktop/DeFish/MINI0016_DeFish.MOV -vf fps=30 /home/local/Desktop/DeFish/Images/MINI0016_DeFish_%d.png

Now we have the working image to test lens effects with “MINI0016_DeFish_37.png”, using ffplay we can preview my results.

ffplay -i /home/local/Desktop/DeFish/Images/MINI0016_DeFish_37.png -vf "lenscorrection=cx=0.5:cy=0.5:k1=-0.227:k2=-0.022"

Here are some links that provide background on the lenscorrection filter.
http://stjhimy.com/posts/2016-10-27-ffmpeg-lens-correction/
http://www.ffmpeg.org/ffmpeg-all.html#lenscorrection

First, we want to establish our center of radial distortion. The values of cx and cy are the x,y coordinate percentages of the center of the radial distortion. Most times this is center, but my image it is not. It is estimated to be at the x,y coordinate 478,394 of an image that has a dimension of 1280×720. This calculates to be 0.38046875,0.547222222 for our cx,cy values respectively. The math is x/1280=cx and y/720=cy

ffplay -i /home/local/Desktop/DeFish/Images/MINI0016_DeFish_37.png -vf "lenscorrection=cx=0.38:cy=0.54:k1=-0.227:k2=-0.022"

Now my correction is more centered for my particular setup. Next we’ll look at the quadratic k1 and double quadratic k2 values. One of the easiest methods I found to determine these values is to script through a range of values and pick out the best results. This script creates a series of images that have the filter applied at different values.

count=1
while [ $count -lt 100 ]
do
ffmpeg -i /home/local/Desktop/DeFish/Script/MINI0016_DeFish_37.png -vf "lenscorrection=cx=0.38:cy=0.54:k1=0.0$count:k2=0.0$count" Pos_MINI0016_DeFish_$count.png
ffmpeg -i /home/local/Desktop/DeFish/Script/MINI0016_DeFish_37.png -vf "lenscorrection=cx=0.38:cy=0.54:k1=-0.0$count:k2=-0.0$count" Neg_MINI0016_DeFish_$count.png
count=`expr $count + 1`
done

From the results, one can pick the closest setting. For me, -.067 did the trick. Now I’ll test my filter on my source video using ffplay

ffplay -i /home/local/Desktop/DeFish/MINI0016_DeFish.MOV -vf "lenscorrection=cx=0.38:cy=0.54:k1=-0.067:k2=-0.067"

Looks good, now lets convert the entire video clip.

ffmpeg -i /home/local/Desktop/MINI0016.MOV -vf "lenscorrection=cx=0.38:cy=0.54:k1=-0.067:k2=-0.067" /home/local/Desktop/DeFish/Final/Final_DeFish_MINI0016.MOV

That should do it. Your source video and fish eye distortion is specific to your rig. When attempting to remove lens distortion, determine the center of the radial distortion first. After you find that, guessing the best quadratic and double quadratic values will be much easier. Using a script is a dirty way to cheat, but it is effective.

I hope you have enjoyed this post and I still think I’ve briefly covered what FFMpeg can do. I look forward to covering more on the topic and hope you can join me.