Skip to content
Demain Docs
Waitlist →

Latency Budget: Per-Stage Budgets and Benchmark Methodology

Every interaction in Portal — from a finger touching the screen to pixels appearing on AR glasses — passes through a multi-stage pipeline. Each stage has a latency budget: a hard ceiling on acceptable processing time. When a stage exceeds its budget, it signals a regression that must be investigated before release. This page documents those budgets, the measurements behind them, and the methodology used to obtain those numbers on the target hardware: the HP EliteBook Ultra G1q (Snapdragon X Elite, X1E80100).

Sources: docs/latency-budget.md#L1-L11, docs/LATENCY_BUDGET.md#L1-L13

Architecture: Five Independent Latency Pipelines

Section titled “Architecture: Five Independent Latency Pipelines”

Portal’s workloads decompose into five latency-critical pipelines, each with distinct performance characteristics — from sub-microsecond serialization in the spatial IPC path to multi-second DSP-bound TTS synthesis.

graph TB
    subgraph "Pipeline 1: Spatial IPC"
        A1[portal-wm encode_request<br/>~80 ns] --> A2[Unix socket write<br/>~10 µs]
        A2 --> A3[portal-spatial process<br/>Wayfire IPC]
        A3 --> A4[decode_response<br/>~60 ns]
    end

    subgraph "Pipeline 2: Streaming"
        B1[DMA-BUF capture] --> B2[EGL UBWC linearize]
        B2 --> B3[IRIS H.265 encode<br/>~5 ms]
        B3 --> B4[RTP packetize<br/>~1.5 µs]
        B4 --> B5[UDP transit 5GHz<br/>~1 ms]
        B5 --> B6[Glasses display]
    end

    subgraph "Pipeline 3: Voice"
        C1[UDP audio recv<br/>~2 ms] --> C2[RNNoise<br/>~1 ms]
        C2 --> C3[VAD DSP<br/>~50 ms]
        C3 --> C4[STT CPU<br/>~200 ms]
        C4 --> C5[NLU MiniLM<br/>~15 ms]
        C5 --> C6[TTS encode+decode<br/>~2.4 s]
        C6 --> C7[RTP to glasses<br/>~1 ms]
    end

    subgraph "Pipeline 4: Touch Input"
        D1[evdev parse<br/>~50 µs] --> D2[FusionEngine<br/>~100 µs]
        D2 --> D3[uinput write<br/>~50 µs]
    end

    subgraph "Pipeline 5: LLM"
        E1[IPC to portal-llmd<br/>~1 ms] --> E2[GenieX model load<br/>~1.6 s cached]
        E2 --> E3[Prompt encode+decode<br/>300-500 ms]
        E3 --> E4[Token generation<br/>~30 tok/s]
    end

The pipelines operate concurrently but independently. Pipeline 2 (streaming) runs continuously at 60 fps regardless of voice or touch activity. Pipelines 3 and 5 share the Hexagon DSP but are arbitrated by the NPU scheduler so that VAD inference preempts LLM workloads (see NPU Scheduling).

Sources: docs/latency-budget.md#L14-L77, portal/streaming/portal_stream.c#L582-L598

Pipeline 1: Spatial IPC — 11 µs Measured, 200 µs Budgeted

Section titled “Pipeline 1: Spatial IPC — 11 µs Measured, 200 µs Budgeted”

The spatial IPC path connects the window manager daemon (portal-wm) to the Wayfire compositor plugin via portal-spatial.so. Messages flow as length-prefixed JSON over Unix domain sockets, with an optional postcard binary serialization layer for homography matrices.

Stage Measured Budget Margin
encode_request (JSON, AssignZone) 82 ns 1 µs 12×
decode_response (JSON, Ok) 59 ns 1 µs 17×
IpcHeader::to_bytes (20-byte struct) 40 ns 500 ns 12.5×
IpcHeader::from_bytes 2.9 ns 100 ns 34×
Postcard serialize IpcMessage 33 ns 1 µs 30×
Postcard parse IpcMessage 26 ns 1 µs 38×
Homography serialize (9×f32) 99 ns 2 µs 20×
Unix socket round-trip (JSON) ~10 µs 100 µs 10×
Total IPC round-trip ~11 µs 200 µs 18×

The dominant cost is the kernel context-switch on the Unix socket — the pure-Rust serialization paths account for less than 1% of total round-trip time. IpcHeader::from_bytes at 2.9 ns is effectively free: it performs four copy_from_slice operations on a #[repr(C)] struct with no allocation.

Sources: benches/benches/spatial_ipc.rs#L7-L133, portal/spatial/src/ipc/message.rs#L44-L96, docs/latency-budget.md#L16-L28

What the Benchmark Measures

Section titled “What the Benchmark Measures”

The criterion harness exercises five distinct scenarios. The encode_request benchmark constructs an IpcRequest::AssignZone variant and calls encode_request() with black_box to prevent the optimizer from eliminating the call. The Unix socket round-trip benchmark is the most end-to-end: it creates a UnixStream::pair(), writes a length-prefixed JSON request from the client side, reads it on the server side, writes back a JSON response, and reads the response on the client — measuring the full kernel-mediated round trip.

The IpcHeader struct uses a fixed 20-byte #[repr(C)] layout (msg_type: u32, payload_size: u32, seq_id: u32, timestamp: u64), so to_bytes() and from_bytes() are pure memcpy operations with zero heap allocation.

Sources: benches/benches/spatial_ipc.rs#L43-L60, portal/spatial/src/ipc/message.rs#L43-L51, portal/spatial/src/ipc/client.rs#L63-L92

Pipeline 2: Streaming — 6 ms Frame-to-Glasses, 20 ms Budgeted

Section titled “Pipeline 2: Streaming — 6 ms Frame-to-Glasses, 20 ms Budgeted”

The streaming pipeline captures Wayland compositor output via DMA-BUF, linearizes QCOM UBWC-compressed buffers through an EGL readback path, encodes to H.265 via the Qualcomm IRIS V4L2 hardware encoder, packetizes into RTP, and transmits over 5 GHz WiFi UDP.

Stage Measured Budget Measurement Method
RtpHeader::to_bytes 28 ns 100 ns criterion
RtpHeader::parse <100 ns 100 ns criterion
RtpBuilder::build_frame (20 ms audio) <500 ns 500 ns criterion
RtpPacket::to_bytes (20 ms audio) <1 µs 1 µs criterion
RTP build + serialize (H.265 small NAL 512B) <2 µs 2 µs criterion
RTP build + serialize (H.265 large NAL 64KB) 1.5 µs 5 µs criterion
Full RTP pipeline (build + serialize + parse) 121 ns 500 ns criterion
IRIS H.265 encode (V4L2) ~5 ms 10 ms GStreamer latency probe
WiFi 5GHz UDP transit ~1 ms 5 ms Packet timestamps
Total frame-to-glasses ~6 ms 20 ms Aggregate

At 60 fps, each frame has a 16.67 ms presentation window. The total frame-to-glasses latency of ~6 ms leaves ~10 ms of headroom, making the streaming pipeline the tightest real-time budget in the system.

Sources: docs/latency-budget.md#L30-L43, portal/streaming/portal_stream.c#L582-L598, benches/benches/rtp_encode.rs#L1-L107

GStreamer Pipeline Construction

Section titled “GStreamer Pipeline Construction”

The streaming daemon constructs its pipeline via gst_parse_launch with the following element chain:

appsrc (DMA-BUF) → queue (leaky=downstream, max-size-buffers=2)
  → videoconvert (BGRx→NV12)
  → v4l2h265enc (IRIS, 8 Mbps CBR, prepend SPS/PPS to IDR)
  → video/x-h265, stream-format=byte-stream, alignment=au
  → rtph265pay (PT=96)
  → udpsink (sync=false, async=false)

The queue element is configured as leaky=downstream with max-size-buffers=2, meaning if the encoder falls behind, the oldest frame is dropped rather than queuing indefinitely. This trades frame loss for latency stability — the alternative (an unbounded queue) would introduce jitter spikes that are worse for AR rendering than occasional dropped frames.

Sources: portal/streaming/portal_stream.c#L582-L598, portal/streaming/portal_stream.h#L1-L19

RTP Constants in the Benchmark

Section titled “RTP Constants in the Benchmark”

The benchmark measures four payload sizes that correspond to real-world scenarios: a 20 ms audio frame at 48 kHz (960 samples × 2 bytes = 1,920 bytes), a 10 ms audio frame (960 bytes), a small H.265 NAL unit (512 bytes, typical for P-frames with low motion), and a large H.265 NAL unit (65,536 bytes, typical for IDR keyframes). The RtpBuilder maintains monotonically incrementing sequence numbers and timestamps with wrapping_add, so the benchmark exercises the same code path as production streaming.

Sources: benches/benches/rtp_encode.rs#L4-L8, portal/stream/src/rtp.rs#L144-L206

Receiver-Side Latency (Informational)

Section titled “Receiver-Side Latency (Informational)”

RTP depacketization and H.265 decode happen on the INMO AIR3 glasses (Android/Unity), not in the Portal Rust workspace. These measurements are included for end-to-end context but are monitored via Android System.nanoTime() logging and the Unity profiler rather than criterion benchmarks.

Stage Typical Latency Measurement Method
RTP depacketize (Java) ~0.5 ms Android System.nanoTime()
H.265 hardware decode (MediaCodec) ~3 ms dequeueOutputBuffer timing
Unity texture upload ~1 ms Unity profiler frame time
Total receive-to-display ~5 ms End-to-end frame timestamp

Sources: docs/RTP_DECODE_BENCHMARK_NA.md#L34-L47

Pipeline 3: Voice — 2.7 s Round-Trip, 5 s Budgeted

Section titled “Pipeline 3: Voice — 2.7 s Round-Trip, 5 s Budgeted”

The voice pipeline is the longest-running latency path in the system. It receives audio from the glasses microphone over UDP, denoises it, detects speech via VAD, transcribes via streaming STT, classifies intent via NLU, synthesizes speech via TTS, and streams the response audio back to the glasses.

Stage Measured Budget Status
UDP audio receive ~2 ms 5 ms ✅
RNNoise denoising ~1 ms 5 ms ✅
VAD (Silero QNN DSP) ~50 ms 100 ms ✅
STT (Sherpa Zipformer CPU) ~200 ms 500 ms ✅
NLU (MiniLM cosine sim) ~15 ms 50 ms ✅
TTS encoder (QNN DSP) ~800 ms 1.5 s ⚠️ DSP-bound
TTS decoder (onnxruntime-qnn) ~1.6 s 2.5 s ⚠️ DSP-bound
RTP audio stream to glasses ~1 ms/frame 5 ms ✅
Total voice round-trip ~2.7 s 5 s ✅

The voice pipeline’s architecture reflects a fundamental hardware constraint: the Hexagon DSP executes the TTS encoder and decoder, but the VAD also needs the DSP. The NPU scheduler resolves this contention by granting VAD inference priority access to DSP compute resources, which means TTS synthesis is interruptible — if the user starts speaking while TTS is running, the pipeline cancels the TTS job and begins a new capture cycle. This is why the budget for TTS decoder is set at 2.5 s (not tighter): the system tolerates the DSP latency because the TTS output is non-interactive audio that can be cancelled.

Sources: docs/latency-budget.md#L45-L57, portal/voice/src/config/defaults.rs#L9-L91

Pure-Rust Voice Micro-Benchmarks

Section titled “Pure-Rust Voice Micro-Benchmarks”

The DSP-bound stages (VAD, TTS encoder/decoder) are measured via production tracing spans. The pure-Rust portions of the voice path — tone generation, SSML parsing, PCM resampling, and PCM format conversion — are measured via criterion micro-benchmarks:

Benchmark Rate / Size Measured
Query tone generation (400→800 Hz sweep) 16 kHz 16.6 µs
Query tone generation 22.05 kHz 22.2 µs
Query tone generation 48 kHz 48.5 µs
Uncertain tone generation (200→400 Hz sweep) 16 kHz 16.1 µs
Uncertain tone generation 22.05 kHz 21.7 µs
Uncertain tone generation 48 kHz 41.7 µs
SSML parse (plain text) — 22 ns
SSML parse (simple <speak>) — 36 ns
SSML parse (complex tags) — 764 ns
PCM resample 22050→48000 (20 ms) — 1.3 µs
PCM resample 22050→48000 (100 ms) — 6.4 µs
PCM f32→s16LE conversion (960 samples) — ~500 ns

Tone generation scales linearly with sample rate because the generator iterates over (sample_rate * 200 / 1000) samples per call. At 48 kHz, 200 ms of audio is 9,600 samples — a single Vec<f32> allocation plus a tight loop computing amplitude * (2π * freq * t).sin() with linear fade in/out at the first and last 5 ms.

SSML parsing is remarkably fast (22 ns for plain text) because the parser performs a single check: if the input does not start with <speak>, the entire string is wrapped in a single SsmlInstruction::Text variant with no further processing. Complex SSML with <prosody>, <break>, <emphasis>, and <say-as> tags takes 764 ns — still well under any practical budget.

Sources: docs/latency-budget.md#L81-L100, benches/benches/voice_tts.rs#L1-L141, portal/voice/src/tts/tones.rs#L7-L46, portal/voice/src/tts/ssml.rs#L71-L80

Pipeline 4: Touch Input — 200 µs Touch-to-Action, 5 ms Budgeted

Section titled “Pipeline 4: Touch Input — 200 µs Touch-to-Action, 5 ms Budgeted”

The touch input path connects an ELAN touchscreen (or virtual pointer) through the FusionEngine to virtual uinput devices that inject events into Wayfire.

Stage Measured Budget Source
evdev Protocol B parse ~50 µs 1 ms Fusion engine bench
FusionEngine (tracker + classifier + actions) ~100 µs 1 ms same
VirtualKeyboard/Pointer uinput write ~50 µs 1 ms same
DRM dumb-buffer blit ~500 µs 2 ms DRM page-flip timing
Total touch-to-action ~200 µs 5 ms aggregate

With a total of ~200 µs touch-to-action latency and a 5 ms budget, this pipeline has the widest margin (25×) of any Portal subsystem. The DRM dumb-buffer blit at ~500 µs dominates the total — it’s a synchronous ioctl-mediated memcpy into a DRM dumb buffer for the keyboard overlay surface.

Sources: docs/latency-budget.md#L59-L67

Pipeline 5: LLM — 500 ms Query, 2 s Budgeted

Section titled “Pipeline 5: LLM — 500 ms Query, 2 s Budgeted”

The on-device LLM path routes queries through portal-llmd to GenieX, which runs llama.cpp inference on the Hexagon DSP.

Stage Measured Budget Source
IPC to portal-llmd ~1 ms 10 ms tracing span
GenieX model load (cached) ~1.6 s 5 s cold-start trace
Prompt encode + decode 300–500 ms 1 s tracing span
Response token generation ~30 tok/s 15 tok/s decode loop
Total LLM query (1 sentence) ~500 ms 2 s aggregate

Model load is excluded from the per-query budget because the KV cache persists across queries within a session. The first query after daemon start incurs the ~1.6 s cold load, but subsequent queries only pay the prompt encode + decode cost. Token generation at ~30 tok/s far exceeds the 15 tok/s floor — a single sentence of ~15 tokens completes in ~500 ms.

Sources: docs/latency-budget.md#L69-L77

Budget Violations and Tightest Margins

Section titled “Budget Violations and Tightest Margins”

As of the X-Elite deployment measurements (2026-07-23), no subsystem violates its latency budget. However, two stages have the tightest margins and warrant monitoring:

Constraint Measured Budget Margin Root Cause
TTS encoder + decoder ~2.4 s combined 4 s combined 1.7× DSP compute is fixed-cost regardless of text length
IRIS H.265 encode ~5 ms/frame 10 ms/frame 2× V4L2 hardware encoder at 8 Mbps CBR, 60 fps

Both are hardware-limited — the DSP compute budget for TTS is bounded by the Hexagon’s inference throughput, and the IRIS encoder latency is a function of the V4L2 driver’s internal buffering. Neither can be improved through software optimization alone; algorithmic changes (e.g., streaming TTS that interleaves encoder and decoder phases) or hardware changes (e.g., zero-copy DMA-BUF to the encoder) would be required.

Sources: docs/latency-budget.md#L104-L113

Benchmark Methodology

Section titled “Benchmark Methodology”

Portal uses a dual measurement strategy: criterion micro-benchmarks for pure-Rust code paths, and production tracing for hardware-bound paths (DSP, V4L2, network).

Criterion Micro-Benchmarks

Section titled “Criterion Micro-Benchmarks”
Parameter Value
Framework criterion v0.5
Samples 100
Warmup 1 second
Measurement time 2–5 seconds
Target hardware Snapdragon X Elite (A76/Oryon cores)
Compilation Release profile, harness = false
Workspace member benches/ (portal-benches crate)

The benches/ workspace member declares three criterion benchmarks, each with harness = false to use criterion’s custom test harness instead of the default Rust test runner. Every benchmark uses black_box() on inputs and outputs to prevent the LLVM optimizer from hoisting or eliminating the benchmarked code.

Sources: benches/Cargo.toml#L1-L34, docs/latency-budget.md#L116-L121

Three Benchmark Binaries

Section titled “Three Benchmark Binaries”
Binary Source File Stages Measured
spatial_ipc benches/benches/spatial_ipc.rs IPC JSON encode/decode, header round-trip, postcard ser/deser, Unix socket round-trip
rtp_encode benches/benches/rtp_encode.rs RTP header ser/parse, audio frame build, H.265 NAL packet build+serialize, full pipeline
voice_tts benches/benches/voice_tts.rs Tone generation (3 sample rates × 2 tones), SSML parse (3 cases), PCM linear resample (2 sizes), PCM f32→s16LE

Sources: docs/latency-budget.md#L131-L139, benches/Cargo.toml#L20-L30

Production Tracing (Non-Benchmarked Paths)

Section titled “Production Tracing (Non-Benchmarked Paths)”

Stages that involve hardware accelerators or external systems cannot be measured via criterion because they require actual DSP compute, V4L2 driver interaction, or network transit. These are measured via:

Path Type Measurement Method
DSP/NPU (VAD, TTS, LLM) tracing spans + journalctl timestamp deltas
Network (UDP to glasses) Sender-side vs receiver-side packet timestamps
V4L2/GStreamer (H.265 encode) rtpjitterbuffer stats from the GStreamer pipeline
DRM (keyboard blit) Page-flip completion timestamps

Sources: docs/latency-budget.md#L116-L121

Running the Benchmarks

Section titled “Running the Benchmarks”

Execute All Benchmarks

Section titled “Execute All Benchmarks”
Terminal window
cargo bench -p portal-benches

This compiles the portal-benches crate in release mode and runs all three benchmark binaries. Criterion generates HTML reports under target/criterion/ with statistical analysis (mean, median, standard deviation, outlier classification).

Build Without Running

Section titled “Build Without Running”
Terminal window
cargo bench --workspace --no-run

Useful in CI to verify that benchmarks compile correctly without spending compute time on actual measurements.

Sources: docs/latency-budget.md#L123-L146

Interpreting Budget Margins

Section titled “Interpreting Budget Margins”

The budget system is designed so that a 2× margin is the minimum acceptable for real-time paths. Margins below 2× indicate that the stage is hardware-limited and cannot be improved without architectural changes. Margins above 10× indicate that the code path is well-optimized and is not a regression risk.

graph LR
    subgraph "Margin Categories"
        SAFE[Safe: 10x+ margin<br/>Not a regression risk]
        MONITOR[Monitor: 2x-10x margin<br/>Track for drift]
        CRITICAL[Critical: below 2x<br/>Hardware-limited]
    end

    SAFE --> |Examples:| S1["IPC encode (12x)<br/>Header parse (34x)<br/>SSML plain text (~1000x)"]
    MONITOR --> |Examples:| M1["Unix socket RT (10x)<br/>Token gen (2x floor)"]
    CRITICAL --> |Examples:| C1["TTS encode+decode (1.7x)<br/>IRIS H.265 (2x)"]

When a regression is detected (a stage that previously passed its budget now fails), the first diagnostic step is to compare the criterion report’s mean against the documented budget. If the mean exceeds the budget but the median does not, the regression is likely caused by outlier behavior (e.g., thermal throttling on the Oryon cores) rather than a code-level regression.

Sources: docs/LATENCY_BUDGET.md#L128-L134, docs/latency-budget.md#L104-L113

Document Lineage

Section titled “Document Lineage”
Document Wave Status Notes
docs/LATENCY_BUDGET.md Wave 14 T177 (2026-07-22) Superseded Retained for historical reference
docs/latency-budget.md Wave 7 items 66-72 (2026-07-23) Canonical Supersedes the above; adds source columns and refines TTS budgets

The canonical document (docs/latency-budget.md) is the authoritative source for all budget numbers. The superseded document (docs/LATENCY_BUDGET.md) contains the same data in a slightly different format with ASCII pipeline diagrams. When budgets change, both files should be updated, but the canonical file is the one CI checks against.

Sources: docs/latency-budget.md#L1-L7, docs/LATENCY_BUDGET.md#L1-L5

Next Steps

Section titled “Next Steps”