A desktop targeting overlay can contain a hundred useful controls or ten thousand. The obvious renderer turns each outline, label background, and glyph into a drawing command. That makes target count visible to the CPU at exactly the wrong rate: every display callback has to walk a scene whose geometry usually did not change.
Saccade sends the renderer targets instead of draw calls. A target is a 48-byte record containing local fixed-point geometry, a resolved label origin, confidence, one style index, and up to sixteen glyph indices. The GPU expands that record into a fixed five-instance shape when the scene changes. Display-rate work updates one active target and issues one indirect draw.
This article derives the representation and follows the bytes for 100 and 10,000 targets. The counts are format arithmetic from the public packet contract. They explain how work scales; they are not a frame-rate claim for a particular GPU or display.
Start with what a target means
The semantic scene uses signed desktop coordinates because a multi-display desktop can extend left or above its origin. The overlay has a narrower job. Each presentation surface already owns one display-local transform, so the packet stores unsigned Q13.3 coordinates: thirteen integer bits and three fractional bits in a 16-bit field.
One eighth of a pixel is enough precision for label and outline placement, and the maximum value covers 8191.875 surface pixels. That includes an 8K-wide display. If a later surface exceeds the range, it must be split or represented by a new packet version. Coordinates never wrap to make an oversized surface appear valid.
This is why position can be two bytes in the overlay while the action scene keeps four-byte signed Q8 coordinates. The two records answer different questions. One places geometry inside a known presentation surface. The other addresses the complete desktop and survives display rearrangement.
4-byte meta
4-byte meta
4-byte meta
4-byte meta
up to 16 glyphs
n * 5 through n * 5 + 4.Five instances are enough
Four instances are thin rectangles covering the target's edges. A fifth instance is one label quad. The label fragment shades its background and samples up to sixteen glyphs from a fixed atlas. It does not emit one instance per character.
The four strips use more vertices than one full target rectangle, but they avoid shading the entire interior. For a large window or text region, fragment coverage matters more than four extra tiny quads. The strips overlap only at their corners and stay nondegenerate because packet admission rejects targets thinner than the outline contract can represent.
The label decision is the larger saving. A conventional text renderer may build a background plus one quad for each glyph. A sixteen-character hint would then require seventeen label-related instances. Saccade resolves the hint and label placement during scene publication, keeps glyph indices in the target record, and shades the complete label from one quad.
An active target gets one additional analytic fill-and-outline instance. That instance is separate because active state can change at 120 Hz while static target geometry stays frozen. No active target means no sixth instance.
static_instances = target_count * 5
total_instances = static_instances + (active_target ? 1 : 0)Separate geometry from metadata
The expanded output is not an array of padded C++ objects. Geometry and metadata use separate buffers. A rectangle is four unsigned 16-bit values, so its stride is 8 bytes. Metadata packs target index, style index, and instance kind into one 32-bit word.
That gives each instance a 12-byte logical output: 8 bytes in the rectangle stream and 4 in the metadata stream. The buffers have aligned bases, but the records do not grow to 16 or 32 bytes merely to make one array-of-structures type convenient.
The separation fits how shaders consume the data. The vertex path needs rectangle coordinates for every instance and a compact metadata load to find the original target and style. The fragment path can read target glyphs only for label instances. Geometry stays dense and linear; labels do not inflate every outline record.
| targets | input packet, one style | instances with active | expanded bytes | three slots |
|---|---|---|---|---|
| 100 | 4,928 | 501 | 6,012 | 18,036 |
| 10,000 | 480,128 | 50,001 | 600,012 | 1,800,036 |
The input formula is 64 + 48N + 64: one 64-byte header, N target records, and one 64-byte style. The expanded formula with an active target is (5N + 1) * 12. Three in-flight slots multiply only the expanded output in this table. The target and style packet remains one immutable source snapshot.
One draw does not mean constant work
The draw count stays at one for 100 and 10,000 targets. That removes per-target CPU submission and state changes. It does not make 50,001 instances cost the same as 501.
Compute expansion scales linearly with target count when a scene changes. The vertex shader processes four vertices per instance. Fragment work depends on covered pixels, overlap, label size, and clipping. Ten thousand tiny nonoverlapping targets can behave differently from one hundred huge overlapping targets even though their raw target counts point in the opposite direction.
This distinction matters when profiling. Instance count predicts record and vertex work. Visible coverage predicts much of the fragment work. Draw count predicts CPU and command submission overhead. Collapsing all three into a single number would make the renderer hard to optimize because a change could improve one axis while regressing another.
Static expansion follows scene epochs
A naive GPU implementation could still repeat expansion every display frame. That moves the loop from CPU code into a compute shader without removing unnecessary work. Saccade instead attaches a scene epoch and transform epoch to the packet. If both match the renderer's retained snapshot, static target bytes are unchanged and expansion is skipped.
A new neural scene, accessibility merge, label assignment, style, glyph atlas, display transform, or topology can publish a new packet. That wakes the display and expands static geometry once. Typing another prefix symbol changes the active target, not the packet.
The display callback therefore writes a small constant block containing active index, inverse surface size, and presentation time. One tiny compute step emits the optional active instance and the indirect instance count. A compute-to-render barrier makes those writes visible, then one instanced draw presents the complete overlay.
per target
time + inverse size
never wait
Three slots, no waiting in the callback
Metal and D3D12 both use three persistent in-flight slots. A slot owns command memory, uniforms, expanded instances, and the completion value that proves those resources may be reused. The display callback selects a retired slot, records work, and submits it.
If all slots remain busy, the callback records a miss and drops that presentation attempt. It does not wait. Waiting would make one slow GPU frame consume the next compositor deadline and could turn temporary pressure into a cascade of late frames.
Metal 4 uses an explicit dispatch-to-render barrier and one command allocator per slot. Metal 3 relies on encoder ordering with the same packet and shader contract. D3D12 performs explicit UAV, shader-resource, and indirect-argument transitions. The API mechanics differ, but the ownership rule is identical: a slot is private until its completion signal retires.
One surface owns one packet
Full desktop coordinates do not enter the overlay shader. Scene publication maps each target through the current desktop-to-surface transform, clips it, and writes a packet for that presentation surface. A target visible on two surfaces can appear in two packets with two local geometries, while the action scene retains one desktop identity.
This also solves mixed scale factors. Font and glyph atlas decisions belong to the surface's scale. Label collision uses the surface-local target set. The active target index is translated through scene-to-overlay and overlay-to-scene maps because clipping can remove different targets on different displays.
Each display can then sleep independently. A scene publication wakes every affected surface for its reveal. A static display with no animated active target has no reason to keep ticking. The 120 Hz capability is available for changing state; it is not a requirement to submit identical transparent frames forever.
The scalar renderer is an oracle
Saccade keeps a plain CPU expansion path. It validates the packet, writes the same rectangles and metadata into caller-provided storage, and reports required capacity before writing. It allocates nothing.
That path is not where I would spend optimization effort. Its purpose is to define exact geometry and metadata for parity checks, to support deterministic replay, and to provide a compatibility implementation. Metal and D3D12 kernels must match it byte for byte before their timing matters.
A general drawing command language would weaken that oracle. Variable instance counts would require prefix sums or CPU offsets. Separate glyph instances would make output size depend on hint length. Arbitrary shapes would turn simple metadata into an interpreter. The fixed five-instance contract is restrictive because the product only needs target outlines, labels, and one active fill.
Reconstructing the draw
The one draw is possible because earlier stages made stronger decisions. Hint assignment resolved label bytes. Collision handling resolved label positions. Surface publication resolved transforms and clipping. The packet fixed target and style layout. Compute expansion fixed each target's output range. Active state remained separate from static geometry.
At display time there is nothing left to discover. The renderer chooses a retired slot, updates active state, writes an indirect count, inserts the platform barrier, and draws. A hundred targets produce 501 instances. Ten thousand produce 50,001. Both use one submission shape, while their actual GPU work remains visible in instance count and covered pixels.
That is the optimization boundary I wanted: target count can scale the work that genuinely depends on targets, without scaling CPU draw calls or repeating scene work at the display rate.