How to merge audio and video using FFmpeg

Introduction

One of our clients needed to showcase their tracks to their clients, while also being able to sync those audio files with video content. They also needed to be able to modify the audio and video volumes so that it would sound good when synced. We needed to build a tool to solve all of these problems and this is where FFmpeg came in to the rescue.

Here's a step-by-step guide on how to merge audio and video using FFmpeg.

Prerequisites

Before we dive into merging audio and video with FFmpeg, make sure you have:

• FFmpeg installed on your system. Check out the official FFmpeg website for installation instructions.
• Your source video file (without audio)
• Your separate audio file
• Basic familiarity with using the command line/terminal

You can verify FFmpeg is properly installed by opening your terminal and running:

ffmpeg -version

Modifying the audio and video volumes

First, we need to modify the audio and video volumes individually before merging them together.

To modify both the audio and video volumes, we can use the following command:

ffmpeg -i {source_file} -af "volume=0.5" -c:v copy {output_video_file}

• -i {source_file}: The source file to modify. For example, input.mp4.
• -af "volume=0.5": The audio filter to modify the volume. 0.5 is 50% volume.
• -c:v copy: The video codec to copy. This flag tells FFmpeg to copy the video stream directly without re-encoding it, which is much faster than re-encoding and preserves the original video quality. The c:v stands for "codec:video".
• {output_video_file}: The output file. For example, output.mp4.

Merging the audio and video

Then, we need to combine those two files together. This is called "muxing".

To do this, we can use this command:

ffmpeg -i {source_video} -i {source_audio} -y -filter_complex "[0:a][1:a]amix=duration=shortest[a]" -map 0:v -map "[a]" -c:v copy {output_video}

Sounds complicated, let's break it down:

• -i {source_video}: The source video file.
• -i {source_audio}: The source audio file.
• -y: Overwrite the output file if it already exists.
• -filter_complex "[0:a][1:a]amix=duration=shortest[a]".
  • [0:a][1:a]: Selects the audio streams from the first input (0:a, the video file) and the second input (1:a, the audio file).
  • amix=duration=shortest: Mixes the two audio streams together. The duration=shortest option ensures that the output audio stream ends when the shorter of the two input audio streams ends.
  • [a]: Names the output of the filter as [a], to be used later in the command.
• -map 0:v: Maps the video stream from the first input (source video) to the output. This means the video remains unchanged.
• -map "[a]": Maps the mixed audio stream ([a]) to the output.
• -c:v copy: Copies the video stream without re-encoding it, preserving its original quality.
• {output_video}: The output file.

Conclusion

This is a basic guide on how to merge audio and video using FFmpeg. With this knowledge, you can now combine separate audio and video files into a single video file, modify the audio and video volumes, and sync them together.