Background removal performance analysis — Citrix ctxbeffect & Fluendo flubkgndremoval

Table of Contents

• 1. Overview
• 2. Benchmark Scope
• 3. Processing Architecture and GStreamer Pipelines
• 4. Measurement Methodology
• 5. Performance Results
• 6. Runtime Stability
• 7. Visual Quality Analysis
• 8. Conclusions

1. Overview

This report compares two background-removal solutions, Citrix ctxbeffect and Fluendo FAIP v1.2.6 flubkgndremoval, focusing on processing performance, latency, resource consumption, and output quality under the same benchmark framework.

FAIP v1.2.6 integrates Raven AI Engine v0.4.7 and provides GPU-native processing capabilities when configured with CUDA inference and Vulkan rendering. In the evaluated GPU-native path, frames are provided through CUDAMemory, processed with Vulkan, inferred with CUDA, and exported as VulkanImage. By keeping compatible stages GPU-addressable, this architecture is designed to reduce unnecessary host/device transitions and take advantage of GPU parallelism.

The objective of this benchmark is to quantify the practical performance characteristics of both solutions and evaluate the processing characteristics of FAIP’s GPU-native architecture under real-time and high-load conditions.

The benchmark evaluates how these architectural differences translate into throughput, processing latency, resource usage, scalability, and visual quality. From a deployment perspective, higher processing capacity and lower latency provide additional headroom for more demanding real-time video-processing workloads and can be relevant for thin-client and high-density video-processing scenarios.

1.1 Executive Summary

This campaign focuses on three representative execution paths covering the main CPU and GPU deployment options evaluated in the benchmark.

Representative execution paths:

Citrix
└── CPU/raw -> CPU processing -> CPU effect -> CPU postprocessing -> CPU/raw

FAIP CPU
└── CPU/raw -> CPU processing -> CPU inference -> CPU postprocessing -> CPU/raw

FAIP GPU-native
└── CUDAMemory -> Vulkan processing -> CUDA inference -> Vulkan postprocessing -> VulkanImage

These three paths provide the main product-level comparison used throughout the Executive Summary: Citrix running on CPU, FAIP running on CPU, and FAIP using its GPU-native CUDA+Vulkan path. In FAIP, processing and postprocessing/render stages can use CUDA or Vulkan depending on the selected GPU-native execution path. The remaining FAIP execution permutations are analyzed in Section 3.

Scenario A and Scenario B answer two complementary questions:

• Scenario A: input is paced to the target 30 or 60 FPS rate, representing real-time operation and reported as Observed FPS and Delivery Ratio.
• Scenario B: pacing is removed to measure maximum processing capacity, reported as Throughput FPS and Headroom.

Scenario A — Real-Time Performance

Scenario A evaluates each execution path with input paced to the target 30 or 60 FPS rate, showing observed behavior under real-time operating conditions.

Real-Time Output

Under paced real-time conditions, FAIP provides stronger output at several demanding workloads. However, the GPU-native path does not reach the target input rate in all tested configurations, with the observed delivery ratio decreasing for some higher-resolution and higher-framerate workloads. This indicates that not all expected frames were observed at the final output during the measurement interval, but does not by itself demonstrate that frames were dropped by the FAIP plugin.

Scenario B — Maximum Processing Performance

Scenario B removes real-time pacing to measure the maximum processing capacity of each execution path and the corresponding plugin latency and resource utilization under load.

Processing Throughput

In unpaced processing, the GPU-native FAIP path provides the largest headroom across the evaluated workloads, showing the practical benefit of the CUDA+Vulkan execution path when real-time pacing is removed.

Direct Plugin Latency

Resource Usage

FAIP GPU-native combines lower direct plugin latency with substantially higher processing headroom, while Citrix maintains a lighter host CPU/RAM footprint across the same workloads.

Visual Quality Analysis

Sample A - Citrix	Sample B - Fluendo

Both solutions provide effective foreground/background separation, but with different visual styles: Citrix emphasizes smoother blur, while FAIP uses opaque/replaced background rendering in this configuration. At a high level, both solutions show similar subject-preservation behavior, with differences mainly in background treatment aesthetics.

1.2 Key Findings

• Maximum processing capacity: FAIP GPU-native delivers the highest Scenario B throughput, reaching 159.50 FPS at 720p30 and 161.50 FPS at 4K60.

• Lower plugin latency: FAIP GPU-native provides the lowest direct plugin latency, with 6.40 ms at 1080p60 and 3.57 ms at 4K60, versus 10.30 ms and 29.02 ms for Citrix.

• Real-time performance: FAIP performs strongly under paced workloads. At 1080p60, FAIP CPU reaches 60 FPS versus 22.50 FPS for Citrix. At 4K60, FAIP GPU-native provides the highest observed output at 29 FPS.

• Better high-load scaling: FAIP GPU-native maintains substantially more processing headroom as resolution and framerate increase, particularly in Scenario B.

• Resource trade-off: Citrix has a significantly lighter CPU/RAM footprint, while FAIP uses more resources in exchange for higher processing capacity and lower GPU-native latency.

• Runtime stability: FAIP still shows teardown instability in some execution paths, which remains an area for further hardening.

Overall, the benchmark shows two distinct deployment profiles. Citrix offers the lighter CPU-oriented footprint, while FAIP GPU-native provides the strongest processing-capacity and latency profile when GPU resources are available. FAIP CPU provides a useful CPU-backed execution option, but at significantly higher host-resource cost. The preferred architecture therefore depends on whether the deployment prioritizes minimum resource footprint or maximum processing headroom and latency performance.

2. Benchmark Scope

2.1 Benchmark Matrix

Workloads: 720p30, 720p60, 1080p30, 1080p60, 1440p30, 1440p60, 4K30, 4K60.

Case counts:

• Scenario A: 72
• Scenario B: 72
• FAIP: 128
• Citrix: 16

2.2 Benchmark Environment

2.3 Evaluated Configurations

Stale theoretical IDs such as faip_cudamemory_cpu_cpu and faip_cudamemory_cpu_cuda are excluded from this corrected final matrix.

2.4 Benchmark Scenarios

For a fixed workload/configuration, both scenarios preserve the same architecture and content family; pacing mode is the principal experimental difference.

2.4.1 Scenario A - Live Stream / Real-Time Performance

Scenario A uses paced input at the target nominal framerate (30 or 60 FPS) to show observed real-time output behavior.

+----------------------+   +----------------------+   +--------------------------+
| Input paced 30/60 FPS|-->| Selected exec path  |-->| Observed: FPS + delivery |
+----------------------+   +----------------------+   +--------------------------+

2.4.2 Scenario B - Offline / Maximum Processing Performance

Scenario B removes input pacing to measure maximum processing capacity and associated latency/resource behavior under load.

+----------------------+   +----------------------+   +----------------------------------+
| Input unpaced        |-->| Selected exec path  |-->| Measured: thrpt+latency+host res |
+----------------------+   +----------------------+   +----------------------------------+

3. Processing Architecture and GStreamer Pipelines

The GStreamer pipeline describes inter-element topology and memory exchange. The FAIP Updated execution plan describes the internal processing/render and AI backend actually selected at runtime.

3.1 Processing Architecture

High-level pipeline architecture:

Input
  |
  v
Decoder
  |
  v
Memory Carrier
  |
  +--------------------------+
  |                          |
  v                          v
CPU/raw                  CUDAMemory
  |                          |
  v                          v
Background Removal Plugin
  |
  v
Output Memory Carrier
  |
  v
Sink

Conceptual plugin internal architecture:

                    Background Removal Plugin
                              |
                              v
                     +------------------+
                     |  Pre-processing  |
                     | CPU/CUDA/Vulkan  |
                     +------------------+
                              |
                              v
                     +------------------+
                     |   AI Inference   |
                     |    CPU / CUDA    |
                     +------------------+
                              |
                              v
                     +------------------+
                     | Post-processing  |
                     | CPU/CUDA/Vulkan  |
                     +------------------+
                              |
                              v
                         Output Frame

3.2 Execution-Path Summary Table

This subsection summarizes the evaluated execution configurations in a compact table.

Scenario A and Scenario B use the same execution-path selection for a given configuration; the primary experimental difference is input pacing mode.

4. Measurement Methodology

4.1 Throughput and Plugin Latency

Throughput uses the final sink rendered-frame rate over the measurement window. Scenario A reports observed FPS with delivery context; Scenario B reports throughput with headroom against nominal workload FPS.

plugin sink timestamp
        |
        v
   background-removal plugin
        |
        v
plugin src timestamp

PluginLatency_i = t_src,i - t_sink,i

Direct plugin latency is measured from matched sink/src PTS using monotonic timestamps. Latency is stage-local and is not equivalent to 1000/FPS or pure CUDA kernel time.

4.2 Resource Measurement and Execution Validation

CPU mean [%], RAM mean [MB], GPU utilization [%], and GPU memory [MB] are sampled in-window. CPU can exceed 100% because process CPU is summed across logical cores.

Execution-path interpretation uses this evidence hierarchy: FAIP Updated execution plan, negotiated caps, AI-device selection, and Vulkan environment. measurement_validity and runtime_stability remain independent dimensions, so VALID + TEARDOWN_ABORT can occur when teardown fails after measurement completion.

5. Performance Results

Primary comparison set in this section:

• Citrix CPU reference: citrix_cpu_cpu_cpu
• FAIP CPU reference: faip_cpuraw_cpu_cpu
• FAIP GPU-native reference: faip_cudamemory_vulkan_cuda

Data-integrity audit policy:

• All cells were verified against testcase-level rows in results_smoke_enriched.json for exact scenario, workload, configuration, validity, FPS, latency, and resources.
• If any primary row were invalid, the table cell would be N/E; no focused rerun values are substituted into the primary matrix.

5.1 Real-Time Performance

Scenario A paced real-time results (Observed FPS with delivery ratio):

5.2 Maximum Processing Performance

Scenario B unpaced maximum processing results (Throughput FPS with headroom):

5.3 Resolution and Framerate Scalability

This subsection interprets scaling trends from Sections 5.1 and 5.2.

In Scenario A (paced), Observed FPS and Delivery Ratio generally decline as resolution and target framerate increase. CPU-reference paths and GPU-native behavior are both workload-dependent, and GPU-native FAIP does not consistently reach the Scenario A target input rate at higher-demand points. Below-target Scenario A output indicates fewer frames observed at final output, but it does not by itself demonstrate plugin-side frame drops.

In Scenario B (unpaced), Throughput FPS and Headroom separate the execution architectures more clearly: CPU-reference paths become constrained earlier at demanding resolution/framerate combinations, while FAIP GPU-native retains substantially more processing headroom across the workload range.

Refer to Section 5.1 for Scenario A detailed values and Section 5.2 for Scenario B detailed values.

5.4 Plugin Latency

Scenario A mean/p95 direct plugin latency (ms), all 8 workloads:

Scenario B mean/p95 direct plugin latency (ms), all 8 workloads:

5.5 Resource Utilization

CPU utilization (%):

RAM (MB):

GPU utilization (%):

GPU memory (MB):

6. Runtime Stability

6.1 Runtime Stability Summary

7. Visual Quality Analysis

7.1 Static Analysis (Reference Frames)

7.1.1 Citrix Reference Frame

7.1.2 FAIP Reference Frame

In static-frame comparison, both solutions show clear human segmentation/matting quality. Citrix appears more strict at subject borders, while FAIP appears slightly smoother in edge treatment. Despite these style differences, final perceived quality is similar.

7.2 Dynamic Analysis (Processed Output Animations)

7.2.1 Citrix Processed Output

7.2.2 FAIP Processed Output

In dynamic clips, the same conclusion holds: both pipelines maintain effective subject separation over time, with Citrix preserving stricter border definition and FAIP keeping a smoother visual transition. Motion behavior appears stable for both in these examples, and overall output quality remains comparable.

8. Conclusions

The benchmark shows that FAIP’s main performance advantage comes from its GPU-native CUDA+Vulkan architecture. When input pacing is removed in Scenario B, faip_cudamemory_vulkan_cuda consistently provides the largest processing headroom and lowest direct plugin latency of the representative execution paths, demonstrating the benefit of keeping video processing, AI inference, and frame exchange on GPU-addressable memory paths.

Under real-time paced conditions (Scenario A), the results are more workload-dependent. FAIP CPU provides strong delivery at several 720p and 1080p workloads, while the GPU-native path does not consistently reach the target input rate despite its much higher maximum processing capacity in Scenario B. At the most demanding 4K60 workload, however, FAIP GPU-native provides the highest observed output of the three representative paths. This difference between Scenario A and Scenario B also shows that maximum processing capacity alone does not guarantee equivalent paced real-time delivery, and that pacing, synchronization, memory flow, and pipeline behavior must be considered together.

The performance advantage of FAIP comes with a clear resource trade-off. Citrix maintains a substantially lighter CPU and RAM footprint, whereas FAIP—particularly its GPU-backed configurations—uses more host and GPU resources. The two solutions therefore represent different deployment profiles: Citrix favors a lightweight CPU-oriented execution model, while FAIP GPU-native favors processing headroom and low plugin latency when GPU resources are available.

From a scalability perspective, the results support the architectural direction of FAIP: as workload complexity increases, the CUDA+Vulkan GPU-native path retains a substantially wider processing envelope than the CPU-reference paths in maximum-load operation. This makes the GPU-native architecture particularly relevant where processing capacity and latency are more important than minimizing resource consumption.

Finally, runtime stability remains the main technical area requiring further hardening. Teardown-abort and memory-corruption signatures were observed even when measurements themselves completed successfully. These issues do not invalidate completed performance measurements, but they should be resolved before treating the evaluated paths as fully mature for long-running production workloads.

Overall, FAIP GPU-native provides the strongest performance profile in this benchmark, particularly for maximum processing capacity and direct plugin latency, while Citrix provides the more resource-efficient baseline. The results also identify two clear priorities for FAIP: improve paced real-time delivery consistency and resolve the observed teardown instability, while preserving the performance advantages of the GPU-native execution path.

8.1 Technical Q&A

Why can CPU usage exceed 100%?

Process CPU usage is aggregated across logical cores, so multi-threaded pipelines can report values above 100%.

Why can Scenario B FPS exceed Scenario A for the same workload?

Scenario A is paced by design, while Scenario B removes pacing to expose maximum processing capacity.

Does CUDA inference alone mean the full pipeline is GPU-native?

No. Input carrier, processing backend, inference backend, and output carrier are distinct dimensions. The runtime execution plan is authoritative.

How should VALID + TEARDOWN_ABORT be interpreted?

It indicates measurement completed successfully, but the process later aborted on teardown.

Are these results long-duration sustainability claims?

No. They are comparative benchmark measurements and are not long-duration endurance evidence.

Ready to put it to the test? Download and try Fluendo AI Plugins 🚀

For further information, get in touch with our team to discuss how we can help optimize your custom GStreamer and GPU pipelines.