Whisper it: FFmpeg 8 can now subtitle your videos on the fly

Media multitool taps Vulkan for GPU encoding, adds VVC support, and dusts off some ancient formats

FFmpeg 8.0 brings GPU-accelerated video encoding via Vulkan – and can now subtitle your videos automatically using integrated speech recognition.

At the start of the week, the FFmpeg project released its eighth major version. It's codenamed "Huffman" after the Huffman code algorithm, which was invented in 1952, making it one of the oldest lossless compression algorithms.

We last looked at FFmpeg version 6.1 in late 2023, in case you need to refresh your memory for what it is and does. Since then, there was 7.0 last April and 7.1 in September, but after an unusually long interval, this is the first release of 2025 – and it's quite a big one.

The changelog lists 30 significant changes, of which the top new feature is integrating Whisper. This means whisper.cpp, which is Georgi Gerganov's entirely local and offline version of OpenAI's Whisper automatic speech recognition model. The bottom line is that FFmpeg can now automatically subtitle videos for you.

This release uses the Vulkan API to do some hardware acceleration of some of its codecs. This includes some FOSS formats, such as encoding AV1 format, and both encoding and decoding FFv1 and the WebM group's VP9 format. It also supports Apple's ProRes RAW. It can handle AV1 files even if they use common encryption (CENC), and supports all types of SCC screen content coding.

This is a little surprising – in a good way – because as The Register said when first covering Vulkan a decade ago, it's a 3D graphics API. Its codename was glNext and it was intended to replace OpenGL one day. An API for hardware-accelerated 3D rendering is not an obvious fit for video coding, but using Vulkan APIs to handle some of the pixel data in video frames means that you get to use the computer's GPU to perform the work, not the CPU – and you do so in a cross-platform sort of way that doesn't even need to know what GPU it's using, let alone being tied to specific GPU models, drivers, or OSes.

FFmpeg now supports VVC – Versatile Video Coding (ITU H.266) – using VAAPI, Intel's open spec for GPU acceleration. As Phoronix reported, this code was contributed by Intel late last year. VVC is the successor to H.265 HEVC, which in turn is the successor to H.264 AVC. That last standard is one of the most widely used video standards, and also widely hardware-accelerated. Huffman can also handle VVC inside Matroska container files.

It's not all about cool new standards, though. For instance, FFmpeg 8 can newly decode RealVideo 6.0, which dates back to at least 2015.

(Incidentally, the same author who wrote that overview of RealVideo 6.0 also has a history of the controversial project lead Michael Niedermayer, whose – presumably temporary – resignation we covered in 2015. We say "presumably temporary" as he tagged this version.)

In 2022, we reported that Google dropped JPEG-XL from Chromium. That doesn't seem to have killed the new format: Huffman can animate displaying JPEG-XL files. It has improved handling of AVP, the Advanced Professional Video format. It can now decode speech encoded in the G.728 low-bandwidth format, and Sanyo's LD-ADPCM format used for WAV files from some Sanyo digital voice recorders. Good news for owners of gadgets such as the ICR-1000 and ICR-B150.

The FFmpeg project – the name means "fast forward MPEG," incidentally – has quite a turbulent history. This has included attempted forks, such as LibAV, of which Clément "ubitux" Bœsch has a history. Reddit also has a pithy executive summary.

LibAV – not to be confused with the FFmpeg library called libav – thrived for a while. Debian switched to it, but switched back again for reasons that LWN explained a decade ago. The rival project was abandoned in 2020.

These days, FFmpeg is an essential tool. We found Wikipedia's list of projects using it eye-opening, and we recommend developer and prolific blogger Drew deVault's article, In praise of ffmpeg. ®