Sports AI
Experience the future of sports with Fluendo’s Sports AI SDK
Bridging multimedia expertise with edge-optimized Machine Learning. We are evolving our core GStreamer foundation into an intelligent Computer Vision pipeline, delivering real-time object tracking and automated video analysis directly to the edge.
AI-powered sports analytics
The project aims to create new products for sports analysis using Artificial Intelligence and Machine Learning optimized for edge devices. These products are a video editor and a drawing tool that can animate video in motion and identify and track objects. This adventure represents a challenge for us, as it is the first time we will apply deep learning to video.
Fluendo has a solid background in multimedia technologies and HW optimisation, and we will complement it with the latest advances in Computer Vision and Artificial Intelligence. Additionally, it will help us establish an AI pipeline that can be reused and improved for future projects and products.
AI-powered sports analytics
The project aims to create new products for sports analysis using Artificial Intelligence and Machine Learning optimized for edge devices. These products are a video editor and a drawing tool that can animate video in motion and identify and track objects. This adventure represents a challenge for us, as it is the first time we will apply deep learning to video.
Fluendo has a solid background in multimedia technologies and HW optimisation, and we will complement it with the latest advances in Computer Vision and Artificial Intelligence. Additionally, it will help us establish an AI pipeline that can be reused and improved for future projects and products.
What has been done
Machine learning operations
Given our emphasis on AI solutions, the MLOps workflow is a cornerstone asset for our company. It streamlines the intricate process of transitioning from data analysis to actionable AI solutions, accelerating our ability to validate and scale novel ideas precisely.
With such a structured and efficient pipeline, we can rapidly iterate on new concepts, validate them in real-world scenarios, and ensure they meet the highest quality and performance standards. This, in turn, enhances our competitive advantage, allows for quicker response to market shifts, and underpins our reputation as leaders in innovation. Investing in our MLOps infrastructure is equivalent to investing in our business’s future-proofing and sustained growth.
What has been done
Machine learning operations
Given our emphasis on AI solutions, the MLOps workflow is a cornerstone asset for our company. It streamlines the intricate process of transitioning from data analysis to actionable AI solutions, accelerating our ability to validate and scale novel ideas precisely.
With such a structured and efficient pipeline, we can rapidly iterate on new concepts, validate them in real-world scenarios, and ensure they meet the highest quality and performance standards. This, in turn, enhances our competitive advantage, allows for quicker response to market shifts, and underpins our reputation as leaders in innovation. Investing in our MLOps infrastructure is equivalent to investing in our business’s future-proofing and sustained growth.
How we do it
Features and capabilities
- 01
Detection
An algorithm designed to identify and locate specific objects, such as a ball or person, or features within an image or video.
- 02
Segmentation
A method that computes pixel-level masks for each object is handy for visualization and visual effects.
- 03
Tracking
Functionality is used to follow the movement of objects across consecutive frames. It can be needed to get tactical insight into any sport.
- 04
Clustering
Groups multiple detected objects into distinct clusters based on their characteristics. It distinguishes players from different teams.
- 05
Calibration
Enables accurate mapping of 2D pixel coordinates to 3D real-world coordinates. It is required for distance calculation.
- 06
HW optimized
Reducing the data copies between the CPU and the GPU will enhance the speed and responsiveness of the inference.
What we did
Our achievements
Model Evolution: From Generic to Specialized
WHERE WE STARTED
- A base model trained on a generic dataset with fundamental detection capabilities.
SPORTS DOMAIN ADAPTATION AND IMPROVEMENT
- With Transfer Learning for domain adaptation, we meticulously refine our model to a target dataset, concentrating on the essential elements like the players and the ball.
RESULT
- Achieved an optimized model with refined and precise detection capabilities.
Threshold Optimization Analysis
EXPLANATION
Comparative analysis (precision vs. recall) when testing our first segmentation algorithm comparing IoU threshold & score threshold.
The best option is to increase the score threshold while maintaining a slightly low IoU threshold.
Tracking Performance & Robustness
RESULTS ANALYSIS
Accuracy: HOTA Score Approximates 0.7 in Object Detection and Association.
Consistent Performance: Error Bars Reflect Tracker’s Uniformity Across Different Sequences.
Robustness: Preserves High Accuracy Across a Range of α Values.
Adaptive Algorithm: Efficiently Manages Variations in Object Localization Accuracy (α)
Multilevel Team Clustering
EXPLANATION
This clustering algorithm has been designed by our RDI team and is based on the usage of a double multilevel clustering algorithm in which, at the first stage, each detected object’s primary colors are computed as main descriptors (with some fine-tuning, confidential data). Later, once each object descriptor is obtained, a higher-level clustering algorithm is added that groups each object in a different cluster (or team). Tested on 20 images, all OK (manual/visual validation) but still needs improvement and an auto evaluation process.
Benchmark: Comparative Model Analysis
EXPLANATION
In our recent analysis, we compared three models: Fluendo’s calibration baselines “real_0”, “big_mix_0” representing Fluendo’s best-performing calibration experiment, and the external “robust” model (only for comparison). “Big_mix_0” outperformed “real_0” in key metrics like MAP and Recall, though with a slightly higher MeanReprojError.
It also surpassed “robust” performance, even with “robust”’s unique architecture and non-commercial constraints. The similarities between “real_0” and “big_mix_0” highlight the advancements in “big_mix_0”’s training, confirming its position as a top-tier homography-based autoencoder AI model.