A desktop targeting overlay can contain a hundred useful controls or ten thousand. The obvious renderer turns each outline, label background, and glyph into a drawing command. That makes target count visible to the CPU at exactly the wrong rate: every display callback has to walk a scene whose geometry usually did not change.
Saccade sends the renderer targets, not a list of draw calls. A target is a 48-byte record containing local fixed-point geometry, a resolved label origin, confidence, one style index, and up to sixteen glyph indices. The GPU expands that record into a fixed five-instance shape when the scene changes. Display-rate work updates one active target and issues one indirect draw. The record is presentation data. Stable action identity remains in the generation-checked scene.
On an M5 Max, the current Metal renderer completes a 10,000-target 4K scene refresh in 0.580 ms p95 and presents the built-in display at 119.810 Hz. Metal and D3D12 expansion also match the scalar packet byte for byte at 1, 100, and 10,000 targets.
A target is surface-local data
The semantic scene uses signed desktop coordinates because a multi-display desktop can extend left or above its origin. The overlay has a narrower job. Each presentation surface already owns one display-local transform, so the packet stores unsigned Q13.3 coordinates: thirteen integer bits and three fractional bits in a 16-bit field.
The packet stores coordinates in Q13.3, giving one-eighth-pixel steps and a range from 0 through 8191.875 surface pixels. That includes an 8K-wide display. One-eighth pixel is the format precision, not the result of a perceptual study. A larger surface must be split or use a new packet version. Coordinates never wrap.
This is why position can be two bytes in the overlay while the action scene keeps four-byte signed Q8 coordinates. The two records answer different questions. One places geometry inside a known presentation surface. The other addresses the complete desktop and survives display rearrangement.
4-byte meta
4-byte meta
4-byte meta
4-byte meta
up to 16 glyphs
n * 5 through n * 5 + 4.Five instances are enough
Four instances are thin rectangles covering the target's edges. A fifth instance is one label quad. The label fragment shades its background and samples up to sixteen glyphs from a fixed atlas. It does not emit one instance per character.
Four strips invoke 16 vertices. One full quad uses four. For an 800 by 600 target with a two-pixel stroke, the full quad covers 480,000 pixels while the four strips cover 5,584 before antialiasing and clipping. That arithmetic motivated the representation, and the 0.580 ms result above measures the complete strip renderer.
For long labels, the larger instance-count reduction comes from generating glyphs only for visible characters inside one label quad. A conventional text renderer may build a background plus one quad for each glyph. A sixteen-character hint would then require seventeen label-related instances. Saccade resolves the hint and label placement during scene publication, keeps glyph indices in the target record, and shades the complete label from one quad.
An active target gets one additional analytic fill-and-outline instance. That instance is separate because active state can change at 120 Hz while static target geometry stays frozen. No active target means no sixth instance.
static_instances = target_count * 5
total_instances = static_instances + (active_target ? 1 : 0)Separate geometry from metadata
The expanded output is not an array of padded C++ objects. Geometry and metadata use separate buffers. A rectangle is four unsigned 16-bit values, so its stride is 8 bytes. Metadata packs target index, style index, and instance kind into one 32-bit word.
The result is 12 bytes per instance.
Eight bytes live in the rectangle stream and four in metadata. The buffers have aligned bases, but the records do not grow to 16 or 32 bytes for the convenience of one array-of-structures type.
That split matches shader access. The vertex shader needs rectangle coordinates for every instance and one compact metadata load for the original target and style. The fragment shader reads glyph data only for label instances. Labels add nothing to each outline record.
| targets | input packet, one style | instances with active | expanded bytes | three slots |
|---|---|---|---|---|
| 100 | 4,928 | 501 | 6,012 | 18,036 |
| 10,000 | 480,128 | 50,001 | 600,012 | 1,800,036 |
The input formula is 64 + 48N + 64: one 64-byte header, N target records, and one 64-byte style. The expanded formula with an active target is (5N + 1) * 12. Three in-flight slots multiply only the expanded output in this table. The target and style packet remains one immutable source snapshot.
One draw does not mean constant work
The draw count stays at one for 100 and 10,000 targets. That removes per-target CPU submission and state changes. It does not make 50,001 instances cost the same as 501.
Compute expansion scales linearly with target count when a scene changes. The vertex shader processes four vertices per instance. Fragment work depends on covered pixels, overlap, label size, and clipping. Ten thousand tiny nonoverlapping targets can behave differently from one hundred huge overlapping targets even though their raw target counts point in the opposite direction.
| 10,000-target operation | machine and surface | result |
|---|---|---|
| scene refresh, Metal 4 GPU complete | M5 Max, 3840 x 2160 private texture | 0.391 ms p50, 0.580 ms p95, 0.697 ms max across 20 process runs |
| active frame, Metal 4 GPU complete | M5 Max, 3840 x 2160 private texture | 0.370 ms p50, 0.462 ms p95, 0.573 ms max across 20 process runs |
| transparent display callback | M5 Max built-in panel, 3024 x 1964 | 119.810 Hz, 1.950 ms worst callback, zero deadline misses across 1,198 frames |
The offscreen profile includes GPU expansion and rendering into three reusable textures. The display-backed profile adds the real display link, drawable acquisition, Metal submission, and presentation. It does not show visible hints or take input.
Static expansion follows scene epochs
A naive GPU implementation could still repeat expansion every display frame. That moves the loop from CPU code into a compute shader without removing unnecessary work. Saccade instead attaches a scene epoch and transform epoch to the packet. If both match the renderer's retained snapshot, static target bytes are unchanged and expansion is skipped.
Any change to the scene or its presentation data can publish a new packet. That wakes the display and expands static geometry once. Typing another prefix symbol changes the active target, not the packet.
The display callback therefore writes a small constant block containing active index, inverse surface size, and presentation time. One tiny compute step emits the optional active instance and the indirect instance count. A compute-to-render barrier makes those writes visible, then one instanced draw presents the complete overlay.
per target
time + inverse size
never wait
Three slots, no waiting in the callback
Metal and D3D12 both use three persistent in-flight slots. A slot owns command memory, uniforms, expanded instances, and the completion value that proves those resources may be reused. The display callback selects a retired slot, records work, and submits it.
If all slots remain busy, the callback records a miss and drops that presentation attempt. It does not wait. Waiting would make one slow GPU frame consume the next compositor deadline and could turn temporary pressure into a cascade of late frames.
The Metal 4 path uses an explicit dispatch-to-render barrier and one command allocator per slot. Metal 3 relies on encoder ordering with the same packet and shader contract. D3D12 performs explicit UAV, shader-resource, and indirect-argument transitions. The API mechanics differ, but the ownership rule is identical: a slot is private until its completion signal retires.
One surface owns one packet
Full desktop coordinates do not enter the overlay shader. Scene publication maps each target through the current desktop-to-surface transform, clips it, and writes a packet for that presentation surface. A target visible on two surfaces can appear in two packets with two local geometries, while the action scene retains one desktop identity.
This also solves mixed scale factors. Font and glyph atlas decisions belong to the surface's scale. Label collision uses the surface-local target set. The active target index is translated through scene-to-overlay and overlay-to-scene maps because clipping can remove different targets on different displays.
Each display can sleep independently. Publishing a scene wakes the affected surfaces for their reveal. On macOS, Saccade pauses the CAMetalDisplayLink callback after the reveal and active animation finish. A new scene or active-state change wakes it again. The 120 Hz loop runs while something is changing. The display callback makes that decision.
The CPU path defines parity
The test expands the same packet on the CPU and compares every rectangle and metadata byte with the GPU result at 1, 100, and 10,000 targets on Metal and D3D12. The scalar path validates the packet, writes into caller-provided storage, and allocates nothing. It is not used by the production GPU renderer.
A general drawing command language would weaken that oracle. The current product needs five static records per target: top, right, bottom, left, and label. The optional active fill is one dynamic record for the whole frame. Adding another static record changes the 5N (+1) output bound and both CPU/GPU parity fixtures.