A media pipeline can stop even when each component is behaving correctly. In Vidarax, one deadlock formed between ffmpeg's input and output pipes; a second failure left published sessions with nobody responsible for cleanup.
Both bugs lived between operations that had to be treated as a pair. The pipe handoff needed an ordering rule, drain before write, while session creation needed ownership to exist before cancellation could interrupt it.
This article follows both failures to the exact state where progress stops, then shows the thread and cancellation boundaries that remove that state. A video stream through Vidarax maps the wider system around them.
The pipes that wedge
One stage talks to ffmpeg over two pipes: encoded video goes into stdin and raw YUV frames come back from stdout. H.264 gives the pair an awkward warm-up. The decoder cannot emit a frame until it has seen parameter sets and a keyframe, so the writer must keep feeding input while nothing comes out.
Lockstep fails first. Write one packet, then read one frame, and the reader waits for a frame that cannot exist yet. Writing freely and reading later only moves the deadlock. Once stdout fills, ffmpeg blocks on its write and stops reading stdin. Then stdin fills and our writer blocks too.
Both programs have obeyed their local logic. Both are asleep forever. The missing property is progress across the pair: before adding more pressure to stdin, our side must make room for whatever ffmpeg has already produced.
The protocol that survives has one dedicated thread and one rule. The thread does nothing but read stdout, assemble complete YUV frames, and push them into a bounded channel with blocking sends. The rule binds the writing side: decode always drains that channel before writing more input. So a full handoff channel can never be the reason the decode call is blocked at stdin, because the writer emptied the channel first; and the reader thread always regains somewhere to put the next frame, so it keeps draining ffmpeg's stdout, and neither pipe can wedge the pair.
Overload has a policy of its own. When frames arrive faster than downstream consumes them, the stage sheds decoded output, keeps the freshest frame, and counts every shed in the exported metrics. It never sheds encoded input, and the reason is mechanical: the decoder is stateful, later frames reference earlier ones, and a dropped input packet corrupts everything until the next keyframe. Dropping a decoded frame costs exactly that frame. The system's job is to say what is happening now, so the policy is freshness over completeness, and the counter shows how often the trade happened.
The boundary between threads and async
The HTTP layer runs on tokio because it holds many mostly idle connections. The media path has the opposite shape: a handful of decode, analysis, and model workers per stream, each continuously CPU-bound or blocked on its neighbor. Those workers use dedicated OS threads joined by bounded channels.
The bounded channels provide backpressure and shutdown. Dropping an upstream sender propagates closure downstream, after which teardown joins the threads. Moving this path onto the async runtime would add cancellation points and executor coupling without an idle population to multiplex.
The two worlds still have to touch, and the touching point is one trait: the event sink, the interface through which the pipeline publishes results. It is synchronous, Send and Sync, callable from a plain thread with no runtime anywhere in sight, and its documentation explains why it is thread-safe instead of asking the reader to trust the trait bounds: event sequence numbers come from a lock-free atomic counter, and the subscriber registry is swapped wholesale behind an atomic pointer, so publishing threads never take a lock. The async side wraps the sink and the sync side calls it. Neither needs to know more about the other than the trait says.
One invariant on the threading is written down at the point of temptation: worker count is clamped to one per stream. H.264 and VP8 decode carry state from frame to frame, and the frame gate's loop-detection state is likewise order-dependent, so splitting one ordered stream across two workers corrupts the output without raising any error. Parallelism happens across sessions only, and the comment saying so sits exactly where a future maintainer would otherwise raise the constant.
The window with no await in it
The async side has its own way of corrupting state. An async handler in Rust is a future, and the caller polling it, here the HTTP server acting for a connected client, can drop it at any await point. The handler does not observe this as an error; it simply ceases to exist between two of its own lines, and any invariant that was going to be restored after the next await never is.
Walk a session-creation handler with that hazard in mind. A client opens a WebRTC ingest session. The handler inserts the new session into the live registry, then spawns the watcher task that owns its lifecycle: monitors it, cleans up after it, frees its stream slot when it ends. Now let the client disconnect at the wrong moment. If the future is dropped after the insert and before the spawn, the registry holds a live session that no task owns. Its stream slot stays occupied forever, and since slots per principal are capped, enough of these ghosts and the server refuses new streams while doing no work at all. Nothing crashes and nothing logs an error; the server just runs out of slots over time.
There is no failed cleanup path to find here, because cleanup never acquired an owner.
The fix is to put no await between registry insertion and watcher spawn. Cancellation can then happen before the session is visible or after its reclaimer exists, but not between those states. A comment at that boundary records why adding an await there would leak the session.
Reinforcements sit on both ends of the lifecycle. On the creation end, session creation runs as a detached transaction, so a client disconnect during creation cannot half-create a run. The test for it is as direct as tests get: spawn the creation, abort the task mid-flight, assert the transaction completed anyway.
On the teardown end, reclaim runs as a single idempotent, cancellation-safe winner with bounded-backoff retry, so the stream slot is freed exactly once no matter how many paths race to free it. A small map of already-reclaimed sessions keeps a late DELETE from tripping over a watcher that already won.
The slot itself is held by an RAII guard whose Drop runs on every exit path, panics included. The release goes through a sharded concurrent map rather than a mutex, so a panicking thread has no lock to poison on the way out, and a panic is not one more way to leak the slot. Together with the closed window, that covers the lifecycle: a session either never becomes visible, or it ends with its slot freed.
These changes produce no speedup to report. Their evidence is direct: the pipe test keeps moving under backpressure, the creation test aborts its caller mid-flight and still observes a complete transaction, and every reclaim path converges on one idempotent winner.