Fluendo Open-Sources TTML Plugin to Empower the GStreamer Community

Since 2012, Fluendo has been at the forefront of implementing HbbTV-related technologies, starting with our initial service for  MStar, a leading SoC manufacturing company in Taiwan. This development led to implementing GStreamer elements for adaptive streaming technologies like MPEG-DASH and the parsing and rendering of TTML (Timed Text Markup Language) subtitles. This innovative work attracted significant interest from Eastern Asian companies like  Seraphic (Shanghai) and MediaTek (Taiwan), leading to a strong demand for Fluendo’s Consulting Services and products. 

Over the following years, the Fluendo and Seraphic collaboration resulted in a combined solution that was fully compliant with HbbTV 2.0.1 and Freeview Play and, later, a compliant Digital TV browser solution for Linux and Android TV. Fluendo’s components became more robust thanks to the Eurofins Ligada test suite, ensuring compliance with the different Digital TV standards, like HbbTV 2.0.2 and EBU-TT-D, among others.

This year, Fluendo made the decision to open-source its TTML plugin, which is now available on the flu-plugins-oss GitHub repository. Although there are no immediate maintenance plans for this plugin, Fluendo anticipates it will be a valuable reference for the GStreamer community to improve its TTML plugin. This step underscores Fluendo’s commitment to advancing TTML support on GStreamer and contributing to the open-source ecosystem.

About Fluendo’s fluttml plugin

Fluendo’s TTML plugin, named fluttml, offers three elements: ttmlparse, ttmlrender, and ttmlsegmentedparse. Developed from scratch, this plugin is entirely independent and does not share any codebase with ttmlsubs. It has been tested with Ligada iSuite and W3C TTML1 test suites. Unlike the GStreamer upstream plugin, the ttmlparse element outputs data in pango-markup format instead of GstSubtitleMeta, making it compatible with elements that accept pango-markup formatted streams, such as textoverlay.

For example, please refer to the plugin documentation for pipelines on how to use this project. 

• ttmlrender: receives subtitles in TTML format and outputs video with subtitles (a video with a transparent background and subtitles rendered on it)
  • Sink: application/ttml+xml
  • Src: video/x-raw
• ttmlparse: transforms a TTML stream into pango-markup (a subtitles format that represents text with rendering parameters)
  • Sink: application/ttml+xml
  • text/x-raw, format=pango-markup
• ttmlsegmentedparse: analyzes subtitles in TTML format, transforming it into a segmented TTML stream (note: a TTML segmented file is an equivalent version of the original TTML file that has been divided into more independent segments).
  • Sink: application/ttml+xml
  • Src: application/ttml+xml,segmented:true

About GStreamer’s upstream plugin

The official upstream GStreamer repository already has a plugin for parsing and rendering TTML called ttmlsubs. It was originally developed by the BBC's R&D department and later integrated into gst-plugins-bad. This plugin is capable of playing back most of the irt-ebu-tt-d-application-samples test samples. However, among other limitations, it cannot playback the official ttml1 samples of the W3C. Here are some of the reasons:

1. It needs a <layout> and <region> tags to be explicitly defined. Otherwise, no text will be displayed, i.e.

	<layout>

	<region xml:id="bottom" tts:displayAlign="after"
	tts:extent="80% 40%" tts:origin="10% 50%"/>
	</layout>
	

2. It does not support a timing format like "5s", it must be something like "00:00:05.000".
3. The <set> tag (useful to change properties at specific times giving the sense of animation) is not supported. 
4. Nested divs are ignored. This can be seen in testsuite/Content/Div001.xml:
5. It does not support foreign namespace   xmlns:f="http://foreign . This can be seen on testsuite/Content/Foreign001.xml

	
	WARN ttmlparse ttmlparse.c:1597:ttml_warn_of_mispositioned_element: 
	Ignoring illegally positioned <div> element.
	

6. In testsuite/Content/Parragraph003.xml it can be observed that the community’s plugin does not support:
   • human-readable colors, i.e. "red", but in hexadecimal format: tts:color="#FF0000"
   • colors on non -<region/> elements, i.e. <p tts:color="#FF0000"> or <span tts:color="#FF0000">.

Conclusion

At Fluendo, we are proud to make this open-source contribution and expect to receive feedback from the GStreamer community. After HYPE, this is another step to initiative is another step in our journey to build a comprehensive open-source plugin collection repository, reinforcing our commitment to innovation and collaboration. For more information, contact our team here.