← Back to docs

Streaming · streaming

Portal Stream — RTP/UDP Network Protocol

Rust network protocol wrapper for the streaming pipeline — RTP packet construction, H.265 NAL unit utilities, UDP streaming. The actual capture and encoding is done by the C-based portal_stream binary.

portal-streamv0.2.0streaming
727
Lines of Code
31
Tests
7
Files

Architecture

Four modules: rtp (RTP packet builder, header construction, payload type management), nal (H.265 NAL unit parsing and fragmentation for RTP transport), udp (UDP socket wrapper with pacing), error (StreamError type). The pipeline: compositor DMA-BUF → GStreamer (glupload → v4l2h265enc IRIS) → rtph265pay → udpsink → glasses IP:5000.

Overview

portal-stream provides the Rust-side network protocol layer for streaming compositor output to AR glasses. It handles RTP (Real-time Transport Protocol) packet construction with proper header fields (sequence numbers, timestamps, SSRC), H.265 NAL (Network Abstraction Layer) unit splitting for fragmentation, and UDP socket management for low-latency transmission. The actual DMA-BUF capture and hardware H.265 encoding is handled by the separate C binary portal_stream (in portal/streaming/), which uses GStreamer with the Qualcomm IRIS V4L2 encoder.

Key Types

RtpBuilderBuilder for constructing RTP packets with proper headers — sequence numbers, timestamps, payload type, SSRC.
RtpHeaderRTP packet header fields — version, payload type, sequence number, timestamp, SSRC, marker bit.
RtpPacketComplete RTP packet — header + payload, ready for UDP transmission.
StreamErrorError type for streaming operations — socket errors, encoding failures, NAL parsing errors.

Modules

rtp

RTP protocol implementation — RtpBuilder for construction, RtpHeader for header manipulation, RtpPacket for complete packets.

nal

H.265 NAL unit utilities — splitting large NAL units into FU-A fragments for RTP transport, NAL header parsing.

udp

UDP streaming — socket management, send pacing to prevent glasses buffer overflow.

error

StreamError enum — socket errors, encoding failures, protocol violations.

Data Flow

Compositor DMA-BUF → GStreamer pipeline → IRIS H.265 encode → RTP packetization → UDP → 192.168.50.255:5000 → INMO AIR3 glasses

Dependencies

External (3)

thiserror workspacehmac workspacesha2 workspace

The C streaming binary (portal/streaming/portal_stream.c) does the actual GStreamer pipeline. This Rust crate provides the protocol layer.