A media pipeline can stop even if each component operates correctly. Vidarax had two failures of this type. In the first failure, a deadlock occurred between the input pipe and the output pipe of ffmpeg. In the second failure, Vidarax published sessions, but no task was responsible for their cleanup.
Each failure needed a rule:
- Pipe rule (order): drain the output (take every output frame that is ready) before you write more input.
- Session rule (ownership): publish a session only after its cleanup task (the watcher) exists.
For the complete media pipeline, read A video stream through Vidarax.
How the two ffmpeg pipes cause a deadlock
One Vidarax stage, the decode stage, connects to ffmpeg through two pipes:
- The decode stage writes encoded video to the stdin of ffmpeg.
- ffmpeg writes raw YUV frames to its stdout, and the decode stage reads them.
The start of an H.264 stream is difficult for this pair of pipes. The decoder cannot output a frame before it receives the parameter sets and a keyframe. The parameter sets give the stream configuration. A keyframe does not refer to earlier frames. Thus, at the start of the stream, the decode stage must continue to write input while ffmpeg writes no output.
Two simple designs fail:
- Lockstep. The decode stage writes one packet and then reads one frame. At the start of the stream, that frame cannot exist yet. The decode stage writes no more input until the read returns. Thus the read never returns.
- Write now, read later. This design only moves the deadlock. These events occur in sequence:
- The stdout pipe becomes full.
- ffmpeg blocks on its write to stdout. While ffmpeg is blocked, it does not read stdin.
- The stdin pipe becomes full.
- The decode stage blocks on its write to stdin.
At that point, each process waits for the other process. Thus the decode stage must obey the pipe rule. Before it writes more data to stdin, it must make room for all the data that ffmpeg already wrote to stdout.
Vidarax uses this design:
- A dedicated reader thread reads stdout. The reader thread sends each decoded frame into a handoff channel.
- The handoff channel holds 16 decoded frames. A send to a full handoff channel blocks.
- Before each write to stdin, the decode stage drains the handoff channel.
This design can still deadlock. If the handoff channel is full, the reader thread blocks. A blocked reader thread does not drain stdout. The drain-before-write order makes the deadlock window smaller. To remove the window, Vidarax needs a bound on the maximum output burst from one input write. The current code has no such bound.
Decoded frames can arrive faster than the next stage can process them. In that case, Vidarax keeps the newest decoded frame and counts each frame that it drops. Vidarax never drops encoded input, because later packets can depend on it.
How a canceled handler can leave a session without an owner
An async Rust handler can be dropped at any await. A dropped handler gets no error. It also cannot finish the work that comes after that await.
Before the fix, the WebRTC session creation handler did two operations in this sequence:
- It inserted the session into the live registry.
- It spawned the watcher. The watcher is the task that monitors the session, cleans it up, and releases its stream slot.
If the client disconnected between these two operations, the handler could be dropped after the insert and before the spawn. The registry then held a session with no watcher.
This orphaned session (a session with no watcher) kept its stream slot forever. Vidarax limits the number of stream slots for each principal. Thus, after enough orphaned sessions, new streams for that principal could not start. At that time, the server did no work. No component crashed, and no component logged an error.
There was no failed cleanup path to find, because no task ever owned the cleanup.
To fix the leak, the handler now has no await between the registry insert and the watcher spawn. Thus a cancellation can occur at two times only:
- before the session is visible in the registry
- after the watcher exists
A cancellation cannot occur between these two states. A comment next to the spawn gives the reason: an await at that point would leak the session.
The creation transaction runs in a detached task, so it is not canceled with the HTTP request. A test starts this transaction. While the transaction runs, the test aborts the handler that awaits it. Then the test checks that the transaction completes.
At teardown, one idempotent reclaim operation releases the stream slot. If more than one caller tries to reclaim the same session at the same time, the callers retry with a bounded backoff. A small map records the reclaimed sessions. This map handles a DELETE request that arrives after the watcher finished.
The slot guard releases its permit in two cases: a normal return, and a panic that unwinds the stack. Vidarax release builds use panic=abort. In that mode, a panic stops the process, and destructors do not run. Thus the process cannot continue with a leaked slot.
Where the blocking threads connect to the async code
Tokio serves many HTTP connections. These connections are idle most of the time. The media workers use dedicated threads, because they are CPU-bound for long periods or they block on ffmpeg and on downstream consumers.
Bounded channels apply backpressure. The channels also carry shutdown.
After an unexpected exit, one supervisor does these actions:
- It closes the peer channels.
- It raises the stop signal.
- It gives the other workers a limited time to finish.
A deadline can show that an OS thread is stuck. But the deadline cannot safely kill that thread. If a worker does not return in the limited time, Vidarax detaches the worker and records a forced shutdown.
Vidarax runs the H.264 and H.265 decoders in an ffmpeg child process. Thus a crash of the native decoder cannot crash the service. Vidarax does not yet have a watchdog for one case: the child process stays alive but makes no progress.
The blocking event sink returns errors directly. Two write paths use a fixed order:
- The write-ahead log of the sink stores an event locally. Then the sink acknowledges the event.
- When Vidarax creates a keyframe, it writes the JPEG sidecar first. Then it appends the metadata that points to the sidecar.
Each stream uses one media worker, for two reasons:
- The H.264 and VP8 decoders keep state from one frame to the next frame.
- Loop detection depends on the order of the frames.
Vidarax gets parallelism from separate sessions: it processes different sessions at the same time.