Blending videos with bash (imagemagick, ffmpeg)

Years ago I was trying to figure out "batch processing." This refers to the concept of applying a single transformation to an entire set of files with one command. Specifically, I wanted to combine two videos, matching frame-by-frame, and blend the two videos together.

The software you'll need to complete this is ffmpeg and imagemagick. Both are CLI softwares you can install via Cygwin, or MacPorts. This is much easier to do via MacPorts, but if you're on Windows, it's not impossible. I was able to install both on an offline Windows 10 box.

To demonstrate what will happen, I'm going to show a single frame as an example:

 This is a blended image, the result of two images composed with an imagemagick CLI command into a final result. To achieve this you run:

$ magick composite -gravity center -blend 30x70% strawberry.png banana.png combo.png

Now, the concept for video, is to blend two videos together, matching frame-by-frame, so that if two 60 fps videos are both 2 seconds in length, you'll blend 120 frames from each into 120 final, blended frames. The concept of completing this task with a single command is what's referred to as "batch processing."

For the code snippets, let's consider "Clouds" and "Flowers" instead of Strawberry and banana, for now...

1. First gather some info about our video, we run:


ffmpeg -i Clouds.MOV

 Metadata:
    major_brand     : qt
    minor_version   : 0
    compatible_brands: qt
    creation_time   : 2019-07-02T21:32:31.000000Z
    com.apple.quicktime.make: Apple
    com.apple.quicktime.model: iPod touch
    com.apple.quicktime.software: 12.0
    com.apple.quicktime.creationdate: 2019-07-02T17:32:31-0400
  Duration: 00:00:05.67, start: 0.000000, bitrate: 15449 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080, 15369 kb/s, 29.98 fps, 29.97 tbr, 600 tbn, 1200 tbc (default)
   
So we have 29.98 fps, let's round up to 30 fps
   
To get the frames we run:
    ffmpeg -i Flowers.MOV -vf fps=30 Flowers_frames/Flowers%03d.jpg -hide_banner

    ...and the same for the Clouds
   
Then to compose them into the final frames we just run this:
   
    $ for i in $(seq -f "%03g" 1 170); do magick composite -blend 50x50% "Clouds_frames/Clouds"${i}".jpg" "Flowers_frames/Flowers"${i}".jpg" "Combo_frames/Combos"${i}".jpg"; done
   
Now that we have the final composed frames, we reanimate these frames into a video with this command:
   
    $ ffmpeg -r 30 -f image2 -s 1920x1080 -i Combo_frames/Combos"%03d".jpg -vcodec libx264 -crf 25 -pix_fmt yuv420p Combo.mov

Note that %05d refers to the length of the largest digit as 5, i.e. 11568 and so all numbers will be padded with the appropriate amount of zeroes to match this length (i.e. 20 becomes 00020), and all filenames will be of the same length. This is so ffmpeg doesn't confuse frame ordering during rendering.

Lastly, you can take the original video's audio and the composed and rendered video and combine those... offsetting the video by .10 seconds (for a better sync)...
ffmpeg -itsoffset 0.1 -i video.mp4 -i original.mp4 -c copy -map 0:v:0 -map 1:a:0 final.mp4

Where video.mp4 is the rendered psychedelic version and original is the original (with audio). The offset component " -isoffset" is applied to the "-i .mp4" that comes directly after it.

That concludes this tutorial. If you have any questions, I'd be glad to answer them. Good luck!

And that's it. I'll post a demonstration once I film the videos properly.