Pipeline processing
End-to-end view of observation processing — from ingest through publishing — with counts and operational controls at every stage.
Ingest
Stage 1Items received8,696
Passed8,696
Validate
Stage 2Items received8,696
Passed8,642
Failed54
Transform
Stage 3Items received8,642
Passed8,628
Transformed8,628
Failed14
Publish
Stage 4Items received8,628
Passed8,616
Transformed8,628
Published8,616
Failed12
Pipeline notes
Validation failures and transform errors are routed to a dedicated replay queue. Reprocessing can be triggered per-stage or per-item with full audit history retained.
All stages emit structured telemetry, supporting per-hub SLOs and per-source availability reporting.