OVERVIEW
A modular research framework that orchestrates four pretrained vision models for soccer analysis: RF-DETR detection, SAM2 segmentation with temporal tracking, SigLIP zero-shot identification, and ResNet jersey classification. A ModelPipeline runs them sequentially, in parallel, or adaptively; a ResultFusion layer reconciles their outputs under seven selectable strategies; and a schema-validated YAML config system with a model registry makes every model swappable by configuration. It is a composition and experimentation harness, built so model combinations can be A/B tested without rewriting code.
ARRIVED AS
Combining several vision models for soccer analysis usually means hard-wiring one fixed pipeline. The goal here was a modular framework where detection, segmentation, identification, and classification models are swappable and configurable, so model combinations can be A/B tested without rewriting code.
Most soccer-vision projects wire one detector to one tracker to one classifier and ship that fixed chain. Research, though, is mostly about swapping pieces: does RF-DETR beat YOLO here, does SAM2 segmentation help identification, is SigLIP good enough to skip a trained roster model? This project is the framework that makes those experiments cheap. It integrates four pretrained models behind one orchestration layer with config-driven presets and pluggable fusion, so a model combination is a config change rather than a rewrite. It is composition and tooling, not a trained end product; the value is the harness.
WHAT I BUILT
- 01An orchestration layer (ModelPipeline) that runs RF-DETR detection, SAM2 segmentation and tracking, SigLIP zero-shot identification, and ResNet jersey classification.
- 02Three execution modes (sequential, parallel, adaptive) plus a result-fusion layer with seven selectable strategies.
- 03YAML-driven, schema-validated config per model, with a model registry and manager so presets are configuration, not code.
WHAT CHANGED
- Swap models or presets through config alone, enabling rapid A/B testing of model combinations.
- SigLIP text-image matching identifies players and teams zero-shot, without training on a specific roster.
- A documented framework with per-model demos and a config system built for research reproducibility.
Data flow
click a stage
Frames are decoded, resized, and cached; the pipeline picks sequential, parallel, or adaptive execution from available resources.
COMPONENT
No component mapped to this stage.
Decisions, with the cost of each.
A decision without its trade-off is marketing. Each row says what was chosen, why, and what it gave up.
A configurable framework instead of one hard-wired pipeline
The research question is which model combination works, so the system was built around a model registry plus validated config presets. Changing detector, fusion strategy, or execution mode is a config edit, not a code change.
More abstraction and moving parts than a single fixed pipeline would need; the payoff is experiment speed.
SigLIP zero-shot identification over a trained roster classifier
Text-image matching identifies players and teams without collecting and labelling per-team training data, so it generalizes to any match out of the box.
Less precise than a model trained on a specific roster; it is a flexible baseline rather than a tuned identifier.
Selectable fusion strategies rather than one fixed rule
Different scenarios reward different fusion: confidence-based when one model dominates, temporal consistency for video, voting when models disagree. Exposing all seven lets the right one be chosen per experiment.
The caller has to know which strategy fits; a wrong choice can fuse worse than a single model.
Sequential, parallel, and adaptive execution modes
Four heavy models do not fit every GPU at once, so the pipeline can run them one at a time, threaded in parallel, or adaptively based on available memory.
Parallel mode adds threading and memory-pressure complexity that sequential mode avoids.
The part that mattered.
The numbers behind the work, and the code that produced them.
- orchestrated
- 4 models
- RF-DETR · SAM2 · SigLIP · ResNet
- fusion strategies
- 7
- voting, weighted, confidence, temporal, spatial, ensemble, adaptive
- execution modes
- 3
- sequential, parallel, adaptive
- config-driven
- YAML
- schema-validated per model
class ExecutionMode(Enum):
SEQUENTIAL = "sequential" # models one after another
PARALLEL = "parallel" # models simultaneously
ADAPTIVE = "adaptive" # choose from available resources
@dataclass
class PipelineConfig:
execution_mode: ExecutionMode = ExecutionMode.SEQUENTIAL
max_workers: int = 4
batch_size: int = 8
enable_gpu_fallback: bool = True
timeout_seconds: int = 60
retry_count: int = 2
memory_limit_gb: float = 8.0
class ModelPipeline:
"""Orchestrates RF-DETR, SAM2, SigLIP, and ResNet with scheduling,
memory management, and fallback strategies."""
The orchestration layer. It runs the four models in whichever mode suits the hardware, with GPU fallback, per-stage timeouts, and retries so one slow or failing model does not take the run down.
class FusionStrategy(Enum):
MAJORITY_VOTING = "majority_voting"
WEIGHTED_AVERAGING = "weighted_averaging"
CONFIDENCE_BASED = "confidence_based"
ENSEMBLE = "ensemble"
TEMPORAL_CONSISTENCY = "temporal_consistency"
SPATIAL_ALIGNMENT = "spatial_alignment"
ADAPTIVE = "adaptive"
# align detections across models, then reconcile labels
from scipy.optimize import linear_sum_assignment
from scipy.spatial.distance import cdist
# cdist -> cost matrix of box-center distances
# linear_sum_assignment -> Hungarian one-to-one match
Fusion is where four independent models become one answer. Spatial alignment matches detections across models with the Hungarian algorithm on box-center distances; temporal consistency then stabilizes identities across frames so labels do not flicker.
class ModelConfigManager:
"""Per-model YAML config with validation schemas and constraints."""
def _init_validation_schemas(self):
self.validation_schemas = {
ModelType.DETECTION: {
"required_fields": ["model_name", "model_path", "input_size"],
"constraints": {
"confidence_threshold": lambda x: 0 <= x <= 1,
"nms_threshold": lambda x: 0 <= x <= 1,
"batch_size": lambda x: x > 0,
},
},
# ... segmentation and classification schemas
}
Each model is loaded from YAML and checked against a schema with field-type and range constraints before it runs. That is what makes a preset safe to swap: bad config fails fast instead of halfway through a video.
✓ LEARNED
For a research harness, the registry and config layer matter more than any single model. Once swapping a detector or fusion rule became a config edit, running experiments got an order of magnitude cheaper.
Zero-shot identification with SigLIP is a useful baseline. It will not beat a roster-trained model, but it works on any match with no labelling, which is exactly what you want early in research.
Fusion is the hard part, not detection. Getting four models to agree (Hungarian alignment across detections, temporal smoothing of identities) was where most of the real engineering went.
Validate config before inference, not during. Schema checks with range constraints turn a class of silent mid-video failures into a fast, obvious error at load time.
◔ NOT DONE YET
- Publish real benchmark outputs (detection, identification, fusion quality) on a fixed clip set so the presets can be compared quantitatively.
- Train a roster-specific jersey/identity head and compare it head-to-head against the SigLIP zero-shot baseline.
- Promote the landing page to a hosted demo that runs an actual clip through the pipeline and shows the fused output.