Open Menu
GStreamer development with Meson and VSCode

GStreamer development with Meson and VSCode

User Name

Written by

Andoni Morales

December 21, 2023

GStreamer is an incredibly powerful and flexible cross-platform open-source multimedia framework that enables developers to create any type of multimedia application, from audio/video players to complex video conferencing software across various hardware and operating systems.

The framework (GStreamer Core and its libraries) and its plugins are mostly written in C, with a small percentage of them using C++ or Objective-C internally. Rust is becoming very popular among GStreamer developers, and most of the new plugins are now written in Rust. The GStreamer code base is quite big, exposing a large API that can be hard to remember and navigate. This is where a good IDE can help during the development process with programmatic language features such as inline hints, jump to definition, hover information, error checking, and so on…

Enters VSCode, an open-source and cross-platform IDE created by Microsoft in 2015 that has quickly become the most popular IDE among developers, as evidenced in StackOverflow’s 2023 Developers Survey. Its support for C/C++ and Rust through extensions and the recently improved support for Meson make it a great IDE for GStreamer development.

Starting with Meson 1.1.0 and the Meson VScode extension 1.9.0, building and debugging GStreamer is as easy as opening the GStreamer sources folder, hitting the ▶️ button, and launching a debug session.

Let’s see it in action on Windows stopping at a breakpoint with a break:

In this blog post, we will explain how to set up VSCode for GStreamer development and how to use the different tools of the IDE seamlessly integrated through the meson extension for a great development experience.

VSCode Introduction

Let’s start with a quick introduction about some VScode fundaments to help us understand how all the pieces are glued together. You can skip this section if you are already familiar with VSCode.

Extensions

Language extensions provide support for languages that are not supported natively, such as C/C++ and Rust. They provide smart editing features such as syntax highlighting, auto-indentation, auto-completion, error checking, and jump to definition, among others. Other extensions provide build system integration, like CMake and Meson. These extensions usually contribute Tasks through the Tasks Provider with the common operations such as configuration, building, or running tests as we will discuss in more detail for the Meson extension.

Tasks

Tasks in VS Code can be configured to run scripts and start processes without entering a command line or writing new code. Workspace or folder-specific tasks are configured from the tasks.json file in the .vscode folder for a workspace. Extensions can also contribute tasks using a Task Provider, and these contributed tasks can add workspace-specific configurations defined in the tasks.json file.

Build system integration through Tasks
Build system integration through Tasks

Command Palette The most important key combination in VS Code is ⇧⌘P (macOS) or ⇧⌃P (Linux, Windows), which opens the Command Palette. This feature provides access to VS Code’s functionality, including keyboard shortcuts for common operations. It is a powerful tool that allows users to navigate and execute commands within the editor quickly.

Meson tasks listed in the command palette
Meson tasks listed in the command palette

Launch configurations

Applications can be run with targets provided by extensions, but most debugging scenarios require creating a launch configuration, such as passing arguments or environment variables. VSCode keeps debugging configuration information in a launch.json. All your launch configurations will appear on the Debug start view, where you can launch them. Here is an example of a simplified GStreamer launch.json that we will describe later in this post:

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "GStreamer - Build and debug",
      "request": "launch",
      "type": "cppdbg",
      "envFile": "${workspaceFolder}/${config:mesonbuild.buildFolder}/meson-vscode.env",
      "program": "${workspaceFolder}/${config:mesonbuild.buildFolder}/subprojects/gstreamer/tools/gst-launch-1.0",
      "args": ["videotestsrc", "!", "autovideosink"],
      "stopAtEntry": false,
      "preLaunchTask": "Meson: Build all targets",
    }
  ]
}

Usage

Installation and setup

The first step to setup our development environment is to install the following extensions:

These extensions will add support for C, C++, and Rust and the integration with Meson.

On Windows, it’s recommended to check out the GStreamer source code in a short path (eg: D:\gstdev) given that environment variables have a maximum length that can be reached for variables such as PATH or GST_PLUGIN_PATH.

The meson project view

The Meson extension provides a view listing all the targets and subprojects that helps to navigate graphically and get an overview of the projects. You can also run the targets from this view with a single click or open the meson.build build file where they are defined.

Meson Project view

Project Configuration

With the Meson extension installed, the first time you open a Meson project, VScode will ask you if you want to configure it with the following options:

  • true: Automatically configure on open
  • false: Never configure on open
  • ask: Ask every time
Meson configuration prompt

These options can be changed in the settings as well. For example, when building GStreamer with subprojects, the configuration process can take up to several minutes and you might prefer to disable it: "mesonbuild.configureOnOpen": false

If you prefer to configure your project manually or need to reconfigure it at any point, the Meson extension provides a “Meson: Reconfigure” task that can be launched anytime from the command palette.

The configure options for your project and the build folder can also be configured in the settings. This is an example of a settings.json:

{
    "mesonbuild.configureOnOpen": false,
    "mesonbuild.buildFolder": "build",
    "mesonbuild.configureOptions": ["-Dlibav=disabled" ,"-Dugly=disabled"],
}

The following video shows the configure task in action:

Language support

Language support in VSCode is done through language extensions, providing the following types of features:

  • Syntax highlighting and bracket matching
  • Smart completions (IntelliSense)
  • Linting and corrections
  • Code navigation (Go to Definition, Find All References)
  • Debugging
  • Refactoring

For GStreamer development, we are interested in C/C++ and Rust which we installed in “Installation and Setup”

C/C++

The extension that provides Intellisense and Debug support for C/C++, is named “C/C++ for Visual Studio Code” and Microsoft maintains it.

For Intellisense to work correctly with our project, it needs to know the following information:

  • Compiler: what compiler we are using and how to configure it
  • Headers: which are the search directories for headers
  • Compile commands: how each of the files in our repository is compiled

Fortunately, the Meson extension does all the heavy lifting for us, both configuring the compiler and headers search path, getting this information from the configuration stage and providing a path to the file with the compile commands that it generated in the configuration stage as well.

The extension is configured with the c_cpp_properties.json configuration file. This is how it would be if we had to do it by hand.

{
  [
    {
      "name": "Win32",
      "compilerPath": "cl.exe",
      "intelliSenseMode": "windows-msvc-x64",
      "includePath": ["${workspaceFolder}/**"],
      "compileCommands": "${workspaceFolder}/${config:mesonbuild.buildFolder}/compile_commangs.json",
      "cStandard": "c17",
      "cppStandard": "c++20",
      "windowsSdkVersion": "10.0.19041.0",
    }
  ],
  "version": 4,
  "enableConfigurationSquiggles": true
}

This is an example with and without compiled commands, where you will see that with compile commands, the code under the __linux__ define is correctly greyed out on Windows.

Without Compile Commands
 

With Compile Commands

Rust Analyzer

The Rust language extensions use rust-analyzer, providing a significant number of features and plenty of configuration settings:

  • Inlay Hints
  • Go to Declaration, Definition, Implementation
  • Macro Expansion
  • Magic completition
  • Much more…

Check out the User Manual for a complete list.

Build

The Meson extension provides a task to build all the targets and one task for each target to build them individually. They can be launched with the Meson: Build task, which opens a dropdown menu with all the available Meson build tasks.

Another way is by using VSCode’s default build task. Pressing ⇧⌘B or running Run Build Task for the first time will show the following picker to select the default build task.

You can now build GStreamer by pressing ⇧⌘B.

Debug

All executable targets that are not tested are exposed as runnable tasks. They can be launched directly from the command palette but do not allow passing parameters or setting environment variables. Most debugging scenarios require a launch configuration (launch.json) allowing us to configure our debugging session with argument parameters and environment variables.

A GStreamer debugging session will usually require to launch a pipeline with gst-launch-1.0 or a sample application created to validate our plugin or scenario.

In this example, our debugging session will run the following pipeline with GST_DEBUG=4:

gst-launch-1.0 d3d11testsrc ! "video/x-raw(memory:D3D11Memory),width=640,height=480" ! dwritetextoverlay text=\"VSCode ❤️‍🔥 GStreamer\" paragraph-alignment=2 ! autovideosink

{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "GStreamer - Build and debug",
      "request": "launch",
      "type": "cppdbg",
      "MIMode": "gdb",
      "windows": {
        "type": "cppvsdbg",
        "program": "gst-launch-1.0.exe",
      },
      "osx": {
        "MIMode": "lldb",
      },
      "envFile": "${workspaceFolder}/${config:mesonbuild.buildFolder}/meson-vscode.env",
      "cwd": "${workspaceFolder}/${config:mesonbuild.buildFolder}/subprojects/gstreamer/tools/",
      "program": "${workspaceFolder}/${config:mesonbuild.buildFolder}/subprojects/gstreamer/tools/gst-launch-1.0",
      "args": [
        "d3d11testsrc", "!",
        "video/x-raw(memory:D3D11Memory),width=640,height=480", "!",
        "dwritetextoverlay text=\"VSCode ❤️‍🔥 GStreamer \" paragraph-alignment=center text-alignment=center", "!",
        "autovideosink"
      ],
      "environment": [
        {
          "name": "GST_DEBUG",
          "value": "4"
        },
      ],
      "stopAtEntry": false,
      "preLaunchTask": "Meson: Build all targets",
    }
  ]
}

Let’s see the import configuration fields of this file:

  • “program”: path to the executable to debug. We set it to the gst-launch-1.0 cli
  • “args”: arguments passed to our program. This is where we define the pipeline we want to run.
  • “environment”: environment variables used to launch our process. We can use this to configure our debug level with GST_DEBUG or to enable pipeline graph dumps with GST_DEBUG_DUMP_DOT_DIR.
  • “envFile”: path to a file with environment variables used to launch our process. We use the meson-vscode.env file generated by Meson, defining all the environment variables required to launch GStreamer from our build directory.
  • “preLaunchTask”: defines a task to launch before starting the debug session. We configure it to launch the task that builds all the targets so that we rebuild GStreamer with all the modifications made and we ensure that we start our debug sessions with a version compiled with the latest changes.

Let’s see it in action on Windows stopping at a breakpoint with a break:

Unit tests

The Meson extension also provides a good integration for tests by providing tasks to run the tests and integrating all the tests in the Tests View. This view provides a very convenient way to interact with tests, allowing us to quickly launch and debug a single test or the whole test suite and visualize the results of the tests.

In this video, we will see how to launch a test from the command palette or directly through the UI:

GStreamer Rust Plugins

Working with the GStreamer Rust Plugins requires a compatible GStreamer version installed, and we’ll need to help rust-analyzer find it by adding PKG_CONFIG_PATH with the path to our GStreamer installation to rust-analyzer.server.extraEnv. We will also add it to rust-analyzer.cargo.extraEnv so that all of our cargo commands use it as well.

"rust-analyzer.server.extraEnv": {
    "PKG_CONFIG_PATH": "D:/gstreamer/1.0/msvc_x86_64/lib/pkgconfig"
},
"rust-analyzer.cargo.extraEnv": {
    "PKG_CONFIG_PATH": "D:/gstreamer/1.0/msvc_x86_64/lib/pkgconfig"
},

After opening the gst-plugins-rs folder in our vscode workspace, the first step is configuring our build and test default tasks. This can be done from the command palette using “Tasks: Configure default build task” and “Tasks: Configure default test task”.

Note that you will also need to add the same PKG_CONFIG_PATH env variable. For some reason, rust-analyzer.cargo.extraEnv does not seem to have an effect.

After configuring the default build and test tasks, our tasks.json file will look like this:

{
	"version": "2.0.0",
	"tasks": [
		{
			"type": "cargo",
			"command": "build",
			"problemMatcher": [
				"$rustc"
			],
			"group": {
				"kind": "build",
				"isDefault": true
			},
			"env": {
				  "PKG_CONFIG_PATH": "D:/gstreamer/1.0/msvc_x86_64/lib/pkgconfig",
			},
			"label": "rust: cargo build"
		},
		{
			"type": "cargo",
			"command": "test",
			"problemMatcher": [
				"$rustc"
			],
			"group": {
				"kind": "test",
				"isDefault": true
			},
			"env": {
				  "PKG_CONFIG_PATH": "D:/gstreamer/1.0/msvc_x86_64/lib/pkgconfig",
			},
			"label": "rust: cargo test"
		}
	]
}

The final step is to create a debug configuration like we did for GStreamer, with the pipeline we want to debug. Since we are using an installed version of GStreamer, we need a bit of extra configuration to use GStreamer:

  • “program”: we need to pass the path to the installed gst-launch-1.0 binary to use.
  • “environment”: we need to add the build directory "${workspaceFolder}/target/Debug" of the rust plugins so that GStreamer can find them.
  • “preLaunchTask”: we will use “cargo build” to ensure our plugins are re-built if we change our source code before launching the debug session.
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "GStreamer RS - Build and debug",
            "request": "launch",
            "type": "cppdbg",
            "MIMode": "gdb",
            "windows": {
                "type": "cppvsdbg",
                "program": "gst-launch-1.0.exe",
            },
            "osx": {
                "MIMode": "lldb",
            },
            "cwd": "/Library/Frameworks/GStreamer.framework/Commands/",
            "program": "/Library/Frameworks/GStreamer.framework/Commands/gst-launch-1.0",
            "args": ["playbin", "uri=gstwebrtc://127.0.0.1:8443?peer-id=68ac29b7-e753-4060-a59b-12f491e05725"],
            "environment": [
                {
                    "name": "GST_PLUGIN_PATH",
                    "value": "${workspaceFolder}/target/Debug"
                },
            ],
            "stopAtEntry": false,
            "preLaunchTask": "cargo build",
        }
    ]
}

Conclusions

Having Intellisense support for GStreamer development on Windows or being able to start a debug session from an IDE was a dream for any GStreamer hacker working on that platform some years ago. The Meson migration that enabled MSVC builds, started to pave the way towards what’s now possible but we still missed a proper IDE integration for a great developer experience.

VSCode coupled with the Meson, C/C++ and Rust extensions has demonstrated that it provides a great environment for GStreamer development.

The Meson extension still has some minor bugs, but they are being solved along the way, and it could very soon be on par with the CMake extension.

Happy Coding!