Convert videos to H.265 / HEVC using FFmpeg and GPU hardware encoding

2021-02-09 2023-12-31 1221 words 6 minutes

I’m using VDR for TV recording. Most public TV channels (via satellite) in Germany that I’m interested in are using MPEG-2 streams with H.264 encoding with a bitrate of about 10000-16000 kb/s. So for an hour of video it could take easily around 6 GByte on disk. This is quite a lot if you want to archive your videos.

I was wondering how to shrink these kind of videos without lowering the “visual” quality too much. And of course it should be fast and maybe even power efficient. First I used my CPU AMD 3900X for this task. This works quite well but uses quite some power. So I knew that AMD RX 5700/6700 (XT) GPU supports H.265 hardware encoding. Without expecting to much I played around a bit and the outcome was surprisingly good - and with way less power needed!

That said I’m by no means a video encoding expert and my main focus just was to keep the same “visual” quality when converting from the MPEG-2 streams in H.264 to H.265 and reduce the size of the original recording by about 50%+. And I also did no audio re-encoding. Audio is copied 1:1.

So what I did to judge about the video quality was comparing two videos side by side with mpv. On the left the original video and on the right the one with the H.265 encoding. I tried various video settings and compared again.

For the impatient here is what I came up with finally after playing around a while with various FFmpeg parameters:

bash

ffmpeg \
  -vaapi_device /dev/dri/renderD128 \
  -i "concat:00001.ts|00002.ts|00003.ts" \
  -vf 'format=nv12,hwupload' \
  -map 0:0 \
  -c:v hevc_vaapi \
  -map 0:a \
  -c:a copy \
  -rc_mode CQP \
  -global_quality 25 \
  -profile:v main \
  -v verbose video.mp4

We’ll talk about the parameters in just a sec. Just to mention it the RX 5700/6700 (XT) supports additional codecs for fast encoding. To figure out for which codecs GPU hardware video encoding acceleration can be used vainfo command provides some information:

bash

vainfo: VA-API version: 1.10 (libva 2.10.0)
vainfo: Driver version: Mesa Gallium driver 20.3.4 for AMD Radeon RX 5700 XT (NAVI10, DRM 3.40.0, 5.10.14-arch1-1, LLVM 11.0.1)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            : VAEntrypointVLD
      VAProfileMPEG2Main              : VAEntrypointVLD
      VAProfileVC1Simple              : VAEntrypointVLD
      VAProfileVC1Main                : VAEntrypointVLD
      VAProfileVC1Advanced            : VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointVLD
      VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
      VAProfileH264Main               : VAEntrypointVLD
      VAProfileH264Main               : VAEntrypointEncSlice
      VAProfileH264High               : VAEntrypointVLD
      VAProfileH264High               : VAEntrypointEncSlice
      VAProfileHEVCMain               : VAEntrypointVLD
      VAProfileHEVCMain               : VAEntrypointEncSlice
      VAProfileHEVCMain10             : VAEntrypointVLD
      VAProfileHEVCMain10             : VAEntrypointEncSlice
      VAProfileJPEGBaseline           : VAEntrypointVLD
      VAProfileVP9Profile0            : VAEntrypointVLD
      VAProfileVP9Profile2            : VAEntrypointVLD
      VAProfileNone                   : VAEntrypointVideoProc

For all entries with a value of VAEntrypointEncSlice hardware encoding can be used and I’m using VAProfileHEVCMain (which is HEVC/H.265 Main profile). As you can see also Main 10 profile is supported if you want to use that one and of course H.264. To give you an impression about encoding speed (H.264 vs H.265): Using the same parameter just with the different encodings I get around 350-400 frames per second (fps) with H.265 and with H.264 I get around 800 fps. So if you’re in a hurry you might prefer H.264 ;-) But for 1080p and even 1280p videos H.265 produces better results and smaller files.

So to get started you obviously need FFmpeg installed. It’s a complete, cross-platform solution to record, convert and stream audio and video (in case you never had heard of it). I’m running Archlinux which means that I’ve normally the latest package versions installed. In case of ffmpeg that means v4.3.1 at the moment. It should be included in every Linux distribution so just use your package manager to install it.

Let’s talk about the FFmpeg parameters used above:

-vaapi_device /dev/dri/renderD128 (which is a shortcut to -init_hw_device vaapi=vaapi0:/dev/dri/renderD128 -filter_hw_device vaapi0) initializes a new hardware device of type vaapi.

VDR stores MPEG-2 transport streams (.ts file suffix) in files using two GByte chunks. So I’ve to concat the files which happens here: -i "concat:00001.ts|00002.ts|00003.ts". -i specifies the input file(s). concat is analogous to using cat on UNIX-like systems. For more information see Concatenate. If your source video file is only one file then -i whatever_file.mp4 works too of course. Btw. not all video files can be concatenated with concat filter. MPEG-2 streams are one of the few options were it works.

-vf 'format=nv12,hwupload' creates the filtergraph and use it to filter the stream. The encoders only accept input as VAAPI surfaces. If the input is in normal memory, it will need to be uploaded before giving the frames to the encoder - in the ffmpeg utility, the hwupload filter can be used for this. It will upload to a surface with the same layout as the software frame, so it may be necessary to add a format filter immediately before to get the input into the right format (hardware generally wants the nv12 layout). The hwupload filter also requires a device to upload to, which needs to be defined before the filter graph is created which I did with -vaapi_device.

Selecting which streams from which inputs will go into which output is either done automatically or with the -map option. So using -map 0:0 -c:v hevc_vaapi as in my example uses the first stream of the original file (which is the video in H.264 encoding) and converts it to H.265 in the destination file (hevc_vaapi means H.265 / MPEG-H part 2 (HEVC)). If you run the command later you’ll see the following debugging output:

bash

Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vaapi))

Next we’ve -map 0:a -c:a copy. This causes to copy the audio streams without modification to the new file. In the output it looks like this:

bash

  Stream #0:1 -> #0:1 (copy)
  Stream #0:2 -> #0:2 (copy)
  Stream #0:3 -> #0:3 (copy)

-rc_mode sets the rate control mode. To see all possible modes ffmpeg -h encoder=hevc_vaapi can be used (this command shows all H.265 VAAPI options btw). These are the available rate control mode’s:

bash

  -rc_mode           <int>        E..V...... Set rate control mode (from 0 to 6) (default auto)
     auto            0            E..V...... Choose mode automatically based on other parameters
     CQP             1            E..V...... Constant-quality
     CBR             2            E..V...... Constant-bitrate
     VBR             3            E..V...... Variable-bitrate
     ICQ             4            E..V...... Intelligent constant-quality
     QVBR            5            E..V...... Quality-defined variable-bitrate
     AVBR            6            E..V...... Average variable-bitrate

Currently the driver only supports CQP, CBR and VBR. After testing a while it turned out that COP works best for me.

The most important parameter for me is -global_quality. The lower the value the better is the video quality but of course also the files are larger. Values between 20 and 25 are the most interesting ones. The default value is 25 which isn’t that bad. For me this rules works best:

If I’ve a video/movie that was produced in HD a value between 21-23 works best and produces good results. But there are also videos/movies that where produced before HD was en vouge. When broadcasted they are scaled to HD e.g. In this case a value between 24-25 works best.

And finally there is -profile:v main. This selects the x265 profile. main should work with every output device. A list of profiles can be found here.

video.mp4 is the output file.

That’s it so far! Happy encoding! 😉

Further links:

Using VAAPI’s hardware accelerated video encoding on Linux with Intel’s hardware on FFmpeg and libav
H.265/HEVC Video Encoding Guide
FFmpeg Hardware/VAAPI