Open-source libva driver with vdpau backend powered by Fluendo

Over a year ago, a client requested support for hardware-accelerated H.264 decoding for devices employing NVIDIA GPUs for CEF (on top of Chromium 97).

Fluendo explored diverse possible pathways and finally reached a solution called flu-va-drivers. From now on, it will be published in GitHub, announced to be open-sourced and licensed as MIT.

CEF is an open-source framework that embeds Chromium browser functionality in other applications. This tool provides a simple and unified API for developing cross-platform desktop and mobile applications using web technologies. It is closely related to the Chromium source code and provides access to many of its features. Several popular products, including Google Chrome, Microsoft Edge, Electron, and others, use the Chromium source code as a base.

Chromium is designed to be a multi-platform browser that supports software decoding of H.264 video using the libffmpeg library and hardware decoding on Linux x86_64 using the libva library (developed by Intel). Nevertheless, Google Chrome did not support VDPAU (created by NVIDIA). The library has a comparable API to VA-API providing hardware acceleration on Linux.

During solution exploration, it was found that an existing project called vdpau-va-driver covered such a gap. It consisted of a driver for libva that implemented the most used VA-API virtual functions, each programmed to call analogous functions in VDPAU. This was already distributed in Ubuntu (Bionic), Debian, and Arch Linux, among other distributions. Combined with Chromium with the VaapiVideoDecoder feature enabled, it would have been the solution.

However, during internal testing, it was found that the driver did not work correctly with Chromium (and derivatives). When tested with Google Chrome (97.0.4692.99) and the VaapiVideoDecoder feature enabled. The web browser fell back to using FFmpegVideoDecoder as the video decoding backend instead of the desired VDAVideoDecoder. 

On the other hand, that did not happen with the Arch Linux package. Only the first part of the Video was rendered. There was an odd scaling behavior, even when not corrected upstream. This would have been fixed by passing an invalid identifier as the background surface when calling the VdpVideoMixerRender function.

Google Chrome using vdpau driver

It must be considered that the newer Google Chrome versions are incompatible with flu-va-drivers. They were not the target versions at that time.

The project that is now open-source can be tested following the instructions documented in the GitHub repository here. Unlike the alternatives, it enabled Google Chrome to render videos in H.264 with 4:2:0 chroma. Subsampling correctly and constrained baseline, main, and high profile.

Newer Google Chrome versions are incompatible with flu-va-drivers, not only because they were not the target versions (that original target version was the 97th) but also because since version 110.0.5481.77, the OpenGL support has been broken, according to reports on Chromium's bug tracker.

Google Chrome using flu-va-drivers-vdpau driver

The flu-va-drivers project has also been tested against Fluster. A multimedia decoding conformance test suite developed by Fluendo, you can read more about Fluster here. Eighty-five of one hundred thirty-five test vectors were successfully decoded. Using a GStreamer-based decoder that utilizes VA-API as a backend (but runs using flu-va-drivers).

On a machine in which decoding originally with a VDPAU-based decoder would have resulted in 114 of 135 test vectors successfully decoded. It means that flu-va-drivers succeeded in 74.56% of the cases the decoder could cover.

Finally, while this project started developing, another similar (nvidia-vaapi-driver) also started developing. Still, Fluendo noticed the existence of this project until the end of flu-va-drivers' development.

This alternative, which leverages from nvdec, would enable Video accelerated decoding on (newer) NVIDIA cards. However, our team could not get successful results on Google Chrome using this driver, nor even with the Chrome-support branch. According to the author, it will never be merged.

You can find the whole flu-va-drivers project here. And if you want to learn more about our FFmpeg Enabler for Chromium, feel free to visit our brand-new website!