GstWASM: The Open-Source project for GStreamer development in the Web

A new edition of Fluendo’s Innovation Days has ended, and it is time to share with you all our advances in the idea we presented last year: GstWASM, GStreamer for the Web. But, before getting into details, we would like to express our appreciation for all the positive feedback we have received from the attendees during the GStreamer Conference at A Coruña last year. Also, thank you to the event organizers for letting us show our work there.

So, what’s new? We aimed to consolidate all efforts into one open-source project, making it easier for developers to collaborate and innovate. We wanted to:

• Track merge requests for the community, including those completed and those still in progress.

• Have proper documentation with guidelines and examples to attract more people to the project.

• Provide practical GstWASM examples of what users can run on their browsers.

• Have packages to help developers create applications using GstWASM.

• Not less important, have a logo to give the project a visual identity!

At Fluendo, we are very proud and excited to make this first public announcement of GstWASM, and we are thrilled to see your feedback. Furthermore, if you can spare just one minute, be sure to take a look at the samples site. Your opinion is invaluable, so don’t hesitate to share your thoughts!

Of course, as with any other project, GstWASM is still a work in progress. There are plenty of things to develop and improve, but this sets the foundation for what will come in the future of web development and browser-based multimedia solutions.

Features

In the end, the core of any project is its features. Let’s review the key features we developed during the Innovation Days.

SDL2 

This was one of the blockages we had last time. We now have a way to show video content on a webpage, as there was no appropriate sink before. We've now overcome this by creating the sdl2sink element

OpenAL

As SDL2 is used for video, OpenAL is used to get an audio sink.  This integration allows developers to render multimedia content with both audio and video working smoothly on web browsers.

Fetch 

We needed to fetch video/audio content from HTTP sources. To do so, we wrapped Emscripten’s Fetch API into a source GStreamer element. 

Canvas

We’ve also created a Canvas API source element to read image buffers from it. This feature is ideal for developers manipulating graphics and image data found on a webpage directly in GStreamer.

OpenGL

We tried OpenGL integration the first time, but we didn’t succeed. This time, we managed to get glimagesink working by implementing a new GstGLContext for Emscripten’s platform. Now, it works using GLES 2.0, but instead of EGL, we directly used Emscripten’s WebGL wrappers.

WebCodecs

We created a whole set of wrappers on top of WebCodecs, similar to what was done in GStreamer for Android’s MediaCodec. Even if it is not complete and the performance of a whole video pipeline is not the best (there is no zero-copy yet), this is an incredible start.

For all those coming to 2024’s GStreamer Conference, we’ll be presenting our results there. It would be great to discuss potential use cases and collaborate further.