RTP Packetization and HMAC-SHA256 Stream Authentication

The Portal platform delivers compositor frames to AR glasses over a wireless link where packet integrity is not optional — a corrupted or forged H.265 NAL unit produces visible artifacts or decoder crashes on the headset. This page documents the two-layer RTP subsystem that handles this responsibility: a C-based GStreamer-integrated packetizer for the video streaming daemon, and a Rust protocol library providing typed RTP headers, H.265 NAL utilities, and an optional HMAC-authenticated UDP transport. Together they form a defense-in-depth authentication model — one layer uses RFC 5285 header extensions for in-band video stream authentication, the other uses truncated HMAC-SHA256 with explicit sequence framing for general-purpose UDP payloads.

Sources: portal/streaming/portal_stream.c#L1-L35 · portal/stream/src/lib.rs#L1-L27

Architecture Overview: Dual Implementation Strategy

The streaming pipeline separates concerns along a language boundary. The C daemon (portal_stream) orchestrates the real-time capture-encode-stream path through GStreamer, where latency budgets measured in single-digit milliseconds preclude FFI round-trips. The Rust crate (portal-stream) provides reusable, type-safe protocol primitives for subsystems that construct or inspect RTP packets at the application layer — voice pipelines, telemetry, test harnesses.

flowchart TB
    subgraph CDaemon["C Streaming Daemon (portal_stream)"]
        DMABUF["Wayland DMA-BUF<br/>zwlr_export_dmabuf_v1"]
        EGL["EGL UBWC Linearizer"]
        APPSRC["appsrc (BGRx)"]
        VCONV["videoconvert"]
        V4L2["v4l2h265enc<br/>(Rockchip MPP HW)"]
        RTPPAY["rtph265pay pt=96"]
        HMAC_PROBE["HMAC Pad Probe<br/>(RFC 5285 two-byte ext)"]
        UDPSINK["udpsink → UDP:5000"]
        
        DMABUF --> EGL --> APPSRC --> VCONV --> V4L2 --> RTPPAY --> HMAC_PROBE --> UDPSINK
    end
    
    subgraph RustCrate["Rust Crate (portal-stream)"]
        RtpBuilder["RtpBuilder<br/>seq/timestamp increment"]
        RtpHeader["RtpHeader<br/>12-byte RFC 3550"]
        RtpPacket["RtpPacket<br/>header + payload"]
        NalUtil["NalUnitType<br/>VPS/SPS/PPS/IDR"]
        UdpStreamer["UdpStreamer<br/>+ udp-hmac feature"]
        
        RtpBuilder --> RtpHeader --> RtpPacket --> UdpStreamer
        NalUtil -.-> RtpPacket
    end
    
    UDPSINK -.->|"RTP/UDP video<br/>to AR glasses"| Glasses["INMO AIR3<br/>Android Receiver"]
    UdpStreamer -.->|"RTP/UDP audio/data"| Consumers["Voice / Telemetry<br/>Consumers"]

The C daemon’s GStreamer pipeline is constructed via gst_parse_launch with a fixed element chain, while the HMAC authentication is injected as a pad probe — a callback that intercepts each buffer just before the network sink, allowing the pipeline topology to remain unchanged whether or not authentication is enabled.

Sources: portal/streaming/portal_stream.c#L582-L598 · portal/streaming/portal_stream.c#L452-L534 · portal/stream/src/lib.rs#L1-L27

C Daemon RTP Pipeline: GStreamer Element Chain

The GStreamer pipeline string is assembled at startup with validated, non-hardcoded network parameters. The element chain follows a linear data flow from raw frames to network packets:

Pipeline Stage	GStreamer Element	Purpose
Input injection	appsrc name=src	Accepts EGL-linearized BGRx frames from DMA-BUF capture
Format conversion	videoconvert	BGRx → NV12 (V4L2 encoder input format)
Hardware encode	v4l2h265enc	Rockchip MPP H.265 encoder with prepend_sps_and_pps_to_idr=1
RTP packetization	rtph265pay pt=96	Converts H.265 byte-stream to RTP packets (PT=96, dynamic)
Network sink	udpsink name=sink	UDP broadcast with configurable bind-address and multicast-iface

The payload type 96 follows the dynamic range convention (96–127 per RFC 3550 §5), with rtph265pay handling NAL unit fragmentation and RTP header construction internally. The encoder’s prepend_sps_and_pps_to_idr=1 flag ensures that each keyframe is self-describing — the Android receiver can begin decoding from any IDR without waiting for out-of-band parameter sets.

Sources: portal/streaming/portal_stream.c#L582-L598

Pipeline Injection Hardening

Before the pipeline string reaches gst_parse_launch, all network tokens (host, port, bind address, multicast interface) pass through validate_safe_token(), which rejects the four shell metacharacters capable of injecting GStreamer pipeline syntax:

static int validate_safe_token(const char *s) {
    if (!s || !*s) return 0;
    for (; *s; s++) {
        if (*s == ';' || *s == '|' || *s == '$' || *s == '`') return 0;
    }
    return 1;
}

This is a defense-in-depth measure: the tokens originate from environment variables (PORTAL_STREAM_TARGET, PORTAL_STREAM_BIND, PORTAL_MULTICAST_IFACE) and argv, all of which are root-controlled on the device. The validator prevents a compromised environment from injecting arbitrary GStreamer elements (e.g., ! filesink location=/etc/passwd) into the pipeline description. The v0.2.1 hardening pass removed all hardcoded deployment IPs — the daemon now exits with STREAM_ERR_CONFIG if no target is supplied.

Sources: portal/streaming/portal_stream.c#L444-L450 · portal/streaming/portal_stream.c#L557-L564 · portal/streaming/portal_stream.c#L677-L685

HMAC-SHA256 Stream Authentication (C Daemon)

Threat Model and Design Rationale

The streaming link operates over UDP broadcast on a local access point (uap0). An attacker on the same network can inject forged RTP packets to disrupt the glasses’ H.265 decoder or replay captured frames. HMAC-SHA256 per-packet authentication defeats both injection and replay: each packet carries a cryptographic tag computed over the full on-wire bytes (header + payload), using a 256-bit shared key that is provisioned out-of-band.

The design prioritizes backward compatibility — receivers that do not implement HMAC validation simply ignore the extension per RFC 3550 §5.3.1. This allows incremental rollout: authenticated glasses validate, legacy glasses still decode.

Sources: portal/streaming/portal_stream.h#L1-L20 · portal/streaming/portal_stream.c#L96-L101

RFC 5285 Two-Byte Header Extension Carriage

The 32-byte HMAC-SHA256 tag is too large for the RFC 5285 one-byte header extension form, which caps each element at 16 bytes (one-byte elements are (len-1) * 4 bytes, maximum 4 words = 16 bytes). The implementation uses the two-byte form instead:

RFC 5285 Form	Profile ID	Max Element Size	Used Here?	Reason
One-byte (0xBEDE)	0xBEDE	16 bytes	No	32-byte tag exceeds 16-byte limit
Two-byte (0x100)	0x100	256 bytes	Yes	Accommodates full HMAC-SHA256 tag

The extension is added via GStreamer’s gst_rtp_buffer_add_extension_twobyte_header, which automatically sets the X bit in the RTP header and inserts the 0x100 extension profile. The application ID is hardcoded as 1 (PORTAL_HMAC_EXT_APPID), matching the glasses receiver’s StreamReceiver.extractHmacExt parser.

Sources: portal/streaming/portal_stream.c#L96-L103 · portal/streaming/hmac_util.h#L1-L14

Pad Probe: Two-Stage Buffer Mapping

The HMAC computation is performed inside a GStreamer pad probe attached to udpsink’s sink pad. This is the final interception point before packets leave the process — the buffer arriving here is the complete RTP packet produced by rtph265pay. The probe uses a two-stage mapping strategy because the two GStreamer APIs serve complementary purposes:

flowchart LR
    subgraph Stage1["Stage 1: Raw Byte Access"]
        A1["gst_buffer_map(buf, GST_MAP_READ)"] --> A2["Read header + payload bytes"]
        A2 --> A3["portal_compute_hmac_sha256(key, data, tag)"]
    end
    
    subgraph Stage2["Stage 2: RTP-Aware Extension"]
        B1["gst_rtp_buffer_map(buf, READWRITE)"] --> B2["gst_rtp_buffer_add_extension_twobyte_header(appid=1, tag, 32)"]
    end
    
    Stage1 -->|"32-byte tag"| Stage2

Stage 1 maps the buffer as a raw GstBuffer for read-only access to the complete on-wire bytes. This is the HMAC input — the exact byte sequence the glasses will receive. Stage 2 maps the same buffer as a GstRTPBuffer with read-write access, which understands RTP structure and can correctly insert the extension (adjusting the X bit, extension profile, and length fields).

The critical invariant is that the HMAC is computed over the pre-extension packet bytes. The extension itself is not included in the HMAC input — it is the authenticator, not the authenticated data.

Sources: portal/streaming/portal_stream.c#L452-L534

Fail-Closed Policy

The pad probe enforces a strict fail-closed policy with three distinct failure modes:

Failure Condition	Probe Action	Rationale
Buffer missing or wrong probe type	Pass through (GST_PAD_PROBE_OK)	Not an RTP packet; no auth obligation
gst_buffer_map (raw read) fails	GST_PAD_PROBE_DROP + log	Cannot read bytes → cannot compute HMAC → must not send
portal_compute_hmac_sha256 returns 0	GST_PAD_PROBE_DROP + log	OpenSSL failure → must not send unauthenticated packet
gst_rtp_buffer_map fails	GST_PAD_PROBE_DROP + log	Cannot add extension → must not send unauthenticated packet

A packet that should have been tagged but wasn’t never reaches the network. This is the core security invariant: when HMAC is enabled, every outgoing packet either carries a valid tag or is silently dropped.

Sources: portal/streaming/portal_stream.c#L462-L534

Key Provisioning and Graceful Degradation

The HMAC key is loaded from the PORTAL_STREAM_HMAC_KEY environment variable, expected as 64 hex characters (32 bytes). The daemon’s startup sequence handles three states:

PORTAL_STREAM_HMAC_KEY State	hmac_enabled	Behavior
Unset or empty	0	Unauthenticated stream (warning logged)
Malformed (wrong length / non-hex)	0	Unauthenticated stream (warning logged)
Valid 64 hex chars	1	Per-packet HMAC authentication

The absence of a key is non-fatal in v0.2.x — this preserves the unauthenticated path for development environments and legacy glasses that lack HMAC validation support. The STREAM_ERR_AUTH error code exists in the type system but is currently unused as a fatal exit code, reserved for future strict-mode enforcement.

Sources: portal/streaming/portal_stream.c#L697-L714 · portal/streaming/portal_stream.h#L12-L18

OpenSSL HMAC Implementation

The portal_compute_hmac_sha256 function wraps OpenSSL’s HMAC() one-shot API, which internally uses EVP_sha256() as the message digest. The implementation is deliberately minimal — OpenSSL 3.0+ deprecates HMAC() in favor of EVP_MAC, but the function remains exported and is the simplest vetted API for single-call HMAC computation.

The input validation enforces a stricter contract than RFC 2104: zero-length keys are rejected (RFC 2104 permits them) because a zero-length key is always a configuration error, never a legitimate runtime state. The NULL-data-with-nonzero-length case is also rejected as a defensive measure, even though OpenSSL’s HMAC() would accept it.

Sources: portal/streaming/hmac_util.c#L20-L42 · portal/streaming/hmac_util.h#L21-L34

Hex Key Decoding

The portal_hex_decode_key function converts the 64-character hex string into 32 raw bytes. It validates length via strlen() (which requires NUL termination) and rejects any non-hex character. The decode is case-insensitive, accepting both lowercase (a-f) and uppercase (A-F) digits. The function writes a constant-time canonical output — the loop processes all 32 bytes unconditionally, though the early-exit on invalid nibbles means it is not truly constant-time over content (acceptable since the key is public-per-network, not a secret being compared).

Sources: portal/streaming/hmac_util.c#L44-L67

Rust RTP Protocol Library

RtpHeader: RFC 3550 Wire Format

The Rust RtpHeader struct provides type-safe construction and parsing of the fixed 12-byte RTP header defined in RFC 3550 §5.1. The serialization produces canonical big-endian output with bitfield packing matching the wire format:

Field	Bit Width	Wire Offset	Notes
Version	2	byte 0, bits 7-6	Always 2
Padding (P)	1	byte 0, bit 5	Not set by builder
Extension (X)	1	byte 0, bit 4	Set by C daemon via pad probe
CSRC count (CC)	4	byte 0, bits 3-0	Always 0
Marker (M)	1	byte 1, bit 7	Set on first packet of talkspurt
Payload type (PT)	7	byte 1, bits 6-0	96 = H.265, 111 = Opus
Sequence number	16	bytes 2-3	Big-endian, wraps at 65535
Timestamp	32	bytes 4-7	Big-endian, wraps at 2³²−1
SSRC	32	bytes 8-11	Big-endian, random per stream

The to_bytes() method returns a fixed-size [u8; 12] array — no heap allocation, no reallocation, deterministic output. The parse() method validates only the minimum length (12 bytes) and does not reject invalid version numbers, preserving forward compatibility with future RTP profiles.

Sources: portal/stream/src/rtp.rs#L10-L109

RtpBuilder: Stateful Packet Construction

The RtpBuilder encapsulates the monotonically-increasing sequence number and timestamp that every RTP stream requires. Each call to build_frame() produces an RtpPacket with the current sequence and timestamp, then advances both using wrapping arithmetic:

self.sequence = self.sequence.wrapping_add(1);
self.timestamp = self.timestamp.wrapping_add(timestamp_increment);

The wrapping semantics match RFC 3550’s specification that sequence numbers and timestamps are unsigned and wrap modulo 2¹⁶ and 2³² respectively. The builder supports configurable initial values via with_initial_sequence() and with_initial_timestamp(), and the timestamp increment is caller-supplied — 960 for 20ms Opus frames at 48kHz, 3600 for H.265 frames at 60fps.

Sources: portal/stream/src/rtp.rs#L144-L217

H.265 NAL Unit Classification

The NalUnitType enum classifies H.265 NAL units by their type field (bits 1-6 of the NAL header byte), enabling the streaming pipeline to distinguish parameter sets from picture data:

NAL Type Code	Enum Variant	Significance
32	Vps	Video Parameter Set — decoder initialization
33	Sps	Sequence Parameter Set — resolution, profile
34	Pps	Picture Parameter Set — slice structure
19, 20	Idr	Instantaneous Decoder Refresh — keyframe
0–9	Trail	Trailing picture (P/B frame)
Other	Other(u8)	Unrecognized type preserved for extensibility

The is_keyframe() and is_parameter_set() methods provide ergonomic predicates used by jitter buffers and session managers to decide when to request keyframe regeneration or buffer resets.

Sources: portal/stream/src/nal.rs#L1-L58

UDP HMAC Feature: Truncated Tag Framing

The Rust crate’s udp-hmac feature provides an alternative authentication model for general-purpose UDP streaming. Unlike the C daemon’s in-band RTP header extension, the Rust approach uses an explicit framing format with a truncated tag:

Wire format: [ seq: 4 bytes BE ] [ payload: N bytes ] [ tag: 16 bytes ]
                                      ↑                        ↑
                              HMAC input ──────────────┘  truncated SHA-256

The HMAC-SHA256 tag covers seq || payload (the 4-byte sequence prefix concatenated with the payload) and is truncated to 16 bytes — half the full 32-byte digest. This truncation follows NIST SP 800-107 guidance that a 128-bit truncated HMAC provides sufficient security margin for most applications while reducing per-packet overhead. The sequence number serves dual purpose: it enables replay detection on the receiver and binds each tag to a specific packet position.

Property	C Daemon (RTP Extension)	Rust Crate (udp-hmac)
Tag placement	In-band (RFC 5285 extension)	Out-of-band (suffix)
Tag length	32 bytes (full)	16 bytes (truncated)
Per-packet overhead	~40 bytes (ext header + tag)	20 bytes (4-byte seq + 16-byte tag)
HMAC input	Full RTP packet bytes	seq ‖ payload
Receiver compat	RFC 3550 §5.3.1 ignore	Requires udp-hmac awareness
Feature gate	Runtime env var	Compile-time Cargo feature

The feature is off by default for incremental rollout. When udp-hmac is disabled, UdpStreamer sends raw payload bytes with no framing overhead, preserving backward-compatible wire format.

Sources: portal/stream/src/udp.rs#L28-L49 · portal/stream/src/udp.rs#L130-L178 · portal/stream/Cargo.toml#L17-L20

Testing and Verification Strategy

The RTP subsystem is covered by four complementary test methodologies, each targeting a different class of failure mode:

C Daemon: cmocka Unit Tests

Three test binaries exercise the C daemon’s authentication and safety primitives. The HMAC tests use RFC 4231 known-answer vectors — the gold standard for HMAC correctness verification:

Test Binary	Test Categories	Key Assertions
test_hmac_util	HMAC correctness, NULL rejection, hex decode	RFC 4231 §4.2 (Test Case 1), §4.3 (Test Case 2) byte-exact match
test_validate_safe_token	Shell injection prevention	Rejects ;, `
test_portal_stream_extended	HMAC boundary, signal handler, NUL termination, hex edges	1-byte key determinism, fail-closed on NULL out, volatile sig_atomic_t contract

The extended tests access static functions by including the full translation unit with #define main _portal_stream_main_disabled — a pattern that links portal_stream.c’s complete dependency tail (Wayland, GStreamer, GBM, DRM, OpenSSL) without executing any of it.

Sources: portal/streaming/tests/test_hmac_util.c#L29-L55 · portal/streaming/tests/test_validate_safe_token.c#L42-L55 · portal/streaming/tests/test_portal_stream_extended.c#L120-L196 · portal/streaming/Makefile#L37-L94

Rust Crate: Property, Integration, and Fuzz Tests

The Rust RTP library employs three testing layers:

Property tests (proptest) verify algebraic invariants that hold for all inputs — the round-trip property (parse(to_bytes(h)) == h) and the size invariant (to_bytes always produces exactly 12 bytes). These tests generate thousands of random header combinations, catching bitfield packing regressions that manual tests might miss.

Integration tests cover the full packet lifecycle: build → serialize → parse → verify, including sequence wrapping at 65535→0, timestamp wrapping at 2³²−1→0, empty payload handling, and 4000-byte large payload preservation. The udp-hmac tests verify the wire format layout (seq ‖ payload ‖ tag), sequence increment, HMAC round-trip verification, and tamper detection — a single flipped bit in the payload must cause verify_truncated_left to fail.

Fuzz testing (libfuzzer) exercises RtpHeader::parse against arbitrary byte input, including truncated buffers, oversized inputs, and adversarial bit patterns in version/padding/CSRC fields. The parser must be panic-free for any byte sequence since it processes untrusted network data.

Sources: portal/stream/src/proptests.rs#L1-L43 · portal/stream/tests/rtp_integration.rs#L1-L213 · fuzz/fuzz_targets/rtp_depacketizer.rs#L1-L23 · portal/stream/src/rtp.rs#L270-L316

Benchmark Coverage

Criterion benchmarks measure serialization throughput across representative workloads:

Benchmark	Payload	Scenario
rtp/header_to_bytes	—	Header-only serialization (12 bytes)
rtp/header_parse	—	Header deserialization
rtp/build_frame_audio_20ms	1920 bytes	Opus 20ms frame @ 48kHz
rtp/build_frame_audio_10ms	960 bytes	Opus 10ms frame @ 48kHz
rtp/build_and_serialize_h265_small	512 bytes	Small NAL unit
rtp/build_and_serialize_h265_large	65536 bytes	Large NAL unit (keyframe-sized)
rtp/full_pipeline_audio_frame	1920 bytes	Build → serialize → parse roundtrip

Sources: benches/benches/rtp_encode.rs#L1-L108

Configuration Reference

C Daemon Environment Variables

Variable	Required	Default	Description
PORTAL_STREAM_TARGET	Yes (v0.2.1)	None	RTP destination IP (broadcast or unicast)
PORTAL_STREAM_BIND	Yes (v0.2.1)	None	Local bind address (uap0 AP interface)
PORTAL_STREAM_HMAC_KEY	No	None	64 hex chars (32-byte HMAC-SHA256 key)
PORTAL_MULTICAST_IFACE	No	uap0	Outbound multicast interface
STREAM_PORT	No	5000	RTP destination port

Rust Crate Feature Flags

Feature	Default	Dependencies Added	Effect
udp-hmac	Off	hmac, sha2	Per-datagram HMAC-SHA256 wrapping in UdpStreamer

Sources: portal/systemd/portal-stream.env.example#L1-L22 · portal/stream/Cargo.toml#L9-L20

Security Properties Summary

The RTP authentication subsystem provides the following guarantees, each verified by the corresponding test category:

Property	Guarantee	Verification Method
Authenticity	Every authenticated packet was produced by a holder of the shared key	HMAC-SHA256 known-answer tests (RFC 4231)
Integrity	Any modification to packet bytes invalidates the tag	Tamper detection test (Rust udp-hmac)
Fail-closed	Authentication failure prevents packet transmission	Pad probe DROP semantics (C extended tests)
Backward compat	Receivers without HMAC support still decode video	RFC 3550 §5.3.1 extension ignore semantics
Injection resistance	Shell metacharacters cannot reach pipeline parser	validate_safe_token cmocka tests
No hardcoded secrets	Deployment IPs and keys are environment-injected	v0.2.1 hardening — no DEFAULT_HOST constant

Next Steps

• Learn how the upstream DMA-BUF capture and H.265 hardware encode feeds into this RTP layer: End-to-End Streaming: DMA-BUF to H.265 Hardware Encode to RTP/UDP
• Understand the latency budget that constrains every microsecond of this pipeline: Latency Budget: Per-Stage Budgets and Benchmark Methodology
• Explore the broader security posture including AppArmor profiles and supply chain audits: Security Hardening: AppArmor, Stream Authentication, and Supply Chain Audits