Testing Strategy: Unit Tests, Property Tests, Fuzzing, and Benchmarks

Portal enforces a four-tier testing pyramid that spans from compile-time lint enforcement through runtime property verification to adversarial input fuzzing. The codebase carries over 6,000 unit tests across 382 modules with #[cfg(test)], 128 proptest macro invocations verifying mathematical invariants, 8 libfuzzer harnesses protecting parsing surfaces against malformed input, and 3 Criterion benchmark suites measuring latency-sensitive hot paths. Every test tier is wired into GitHub Actions CI, with Miri UB-scanning on the FFI-heavy crates and a separate on-device smoke suite for the Snapdragon target. This architecture ensures that protocol parsers, IPC framing, and rendering math are validated not just for correctness on expected inputs, but for panic-free robustness on any byte sequence an adversary could inject.

Testing Architecture Overview

The testing strategy follows a layered approach where each tier addresses a distinct class of failure:

graph TD
    subgraph "Compile-Time Gate"
        LINTS["Workspace Lints<br/>correctness, suspicious, style,<br/>perf, pedantic = deny"]
        CLIPPY_PROD["cargo clippy --lib --bins<br/>Zero-tolerance production gate"]
        CLIPPY_TEST["cargo clippy --tests<br/>Permissive test gate"]
    end

    subgraph "Unit Tests — 6,063 tests"
        INLINE["Inline #[cfg(test)] modules<br/>382 files"]
        INTEG["Integration tests/<br/>223 .rs files"]
        ASYNC["Tokio async tests<br/>624 functions"]
    end

    subgraph "Property Tests — 128 proptest! blocks"
        ROUNDTRIP["Serialization roundtrips"]
        INVARIANTS["Mathematical invariants<br/>(score bounds, velocity ≥ 0)"]
        PARSER["Parser robustness<br/>(evdev, IPC, .desktop)"]
    end

    subgraph "Fuzzing — 8 libfuzzer targets"
        FUZZ_WS["fuzz/ (5 targets)<br/>RTP, IPC, .desktop, TOML, shlex"]
        FUZZ_CRATE["Per-crate fuzz packages<br/>input, spatial, pcp-stream"]
    end

    subgraph "Benchmarks — Criterion"
        BENCH_HOT["benches/ crate<br/>RTP encode, IPC roundtrip, TTS"]
        BENCH_CRATE["Per-crate benches<br/>voice, launcher, context, style"]
    end

    subgraph "CI Safety Nets"
        MIRI["Miri UB scan<br/>npu-runtime, spatial, input, voice"]
        COVERAGE["llvm-cov ≥ 85% line gate"]
        SMOKE["HP Spaceboard smoke<br/>on-device daemon lifecycle"]
    end

    LINTS --> CLIPPY_PROD
    CLIPPY_PROD --> UNIT
    CLIPPY_TEST --> UNIT
    UNIT[""] --> INLINE
    UNIT --> INTEG
    UNIT --> ASYNC
    INLINE --> ROUNDTRIP
    INLINE --> INVARIANTS
    INLINE --> PARSER
    PARSER --> FUZZ_WS
    PARSER --> FUZZ_CRATE
    ROUNDTRIP --> BENCH_HOT
    INVARIANTS --> BENCH_CRATE
    FUZZ_WS --> MIRI
    FUZZ_CRATE --> COVERAGE
    BENCH_HOT --> SMOKE

Sources: Cargo.toml#L166-L179, .github/workflows/test.yml#L1-L26, .github/workflows/check.yml#L1-L87, .github/workflows/miri.yml#L1-L51

Unit Tests: Inline and Integration

Unit tests in Portal follow the standard Rust convention of two physical locations: #[cfg(test)] modules embedded inside src/ source files (for private-item access) and standalone .rs files under tests/ (for public-API integration scenarios). The inline pattern is the dominant form, with 382 source files containing test modules. Many crates use a hierarchical src/.../tests/ directory structure where each submodule owns a mod.rs aggregator with child test files — for example, the launcher crate has 30+ test files organized by provider (providers/app/tests/, providers/file/tests/, providers/system/tests/) and subsystem (search_engine/tests/, action_registry/registry/tests/).

The integration tests in tests/ exercise cross-module contracts through the crate’s public API. For the streaming pipeline, portal/stream/tests/rtp_integration.rs#L1-L213 validates the complete RTP packet lifecycle — build → serialize → parse → verify — across 12 test functions covering field roundtrips, sequence wrapping at the u16 boundary, timestamp wrapping at u32::MAX, payload preservation for all 256 byte values, and rejection of short/empty buffers. Async tests use #[tokio::test], with 624 such functions across the workspace, primarily in the launcher, context, and voice crates where daemon lifecycle and event ingestion require async runtime.

Test Category	Location	Count	Access Scope
Inline unit tests	`src//tests.rs` or `src//tests/mod.rs`	382 files	Private + public items
Integration tests	`tests/*.rs`	223 files	Public API only
Async tests	Either location	624 `#[tokio::test]`	Event loops, daemon lifecycle
Property tests	Either location	128 `proptest!` blocks	Invariant verification

The npu-runtime crate demonstrates the inline-private pattern particularly well. Its portal/npu-runtime/src/tests.rs#L1-L80 exercises page_round, MAX_DMA_ALLOC, and PAGE_SIZE — all pub(crate) or private items — with exhaustive boundary tests: zero, below-one-page, exact-multiples, and overflow cases. The hardware-dependent tests (real ioctl against /dev/fastrpc-cdsp-secure) are separated into tests/dma_allocation.rs with #[ignore] gating.

Sources: portal/stream/tests/rtp_integration.rs#L23-L51, portal/npu-runtime/src/tests.rs#L1-L11, portal/spatial/tests/ffi_tests.rs#L1-L40

Lint Enforcement: Two-Tier Clippy Gate

Portal’s workspace-level lint configuration represents a strict separation between production and test code quality standards. Production code (--lib --bins) must pass clippy with zero warnings across correctness, suspicious, style, complexity, performance, and pedantic categories — all promoted to deny. Additionally, unwrap_used, expect_used, and panic_in_result_fn are denied in production code, enforcing the architectural rule that all errors must surface via Result.

Test code is held to a deliberately permissive standard because tests legitimately use unwrap()/expect() as assertion primitives — panicking on setup failure is the test signal. The CI clippy command for tests explicitly allows clippy::unwrap_used, clippy::expect_used, clippy::as_conversions, and dead-code lints, with inline justification comments documenting the rationale:

# Production gate — strict.
- run: cargo clippy --lib --bins --workspace --all-features -- -D warnings
# Test-only gate — deliberately permissive.
- run: cargo clippy --tests --workspace --all-features -- -D warnings \
    -A clippy::unwrap_used -A clippy::expect_used ...

The workspace lint block also enforces await_holding_lock = "deny" — catching a deadlock pattern where a MutexGuard is held across an .await point — and mandates # Errors and # Safety doc sections on all public functions returning Result or containing unsafe blocks.

Sources: Cargo.toml#L166-L200, .github/workflows/check.yml#L28-L46

Property Tests: Invariant Verification at Scale

Portal uses proptest 1.4 (declared as a workspace dependency) to verify mathematical and structural invariants that must hold for all valid inputs, not just hand-picked examples. The workspace contains 128 proptest! macro invocations across 8 crates, organized by the type of invariant being tested. Each proptest module generates random inputs via composable strategies, exercises the system under test, and asserts properties using prop_assert! / prop_assert_eq! — the shrinking infrastructure automatically minimizes any counterexample to the simplest failing input.

The launcher crate’s portal/launcher/tests/property_tests.rs#L1-L73 is the most comprehensive property test suite, defining custom proptest strategies that generate syntactically valid ActionId strings (three or more dot-separated [a-z0-9_] components). Five property groups are verified: (1) all generated valid IDs parse successfully, (2) new(s).unwrap().to_string() == s roundtrip, (3) component splitting preserves all segments, (4) strings with fewer than 3 components always fail, and (5) uppercase and special characters always cause rejection.

The fusion engine’s portal/input/tests/proptest_fusion.rs#L153-L199 demonstrates physics-domain property testing: it generates random touch start/end positions and time deltas, then verifies that velocity computations never produce NaN or negative-infinity values — because the internal formula sqrt(dx² + dy²)/dt is mathematically guaranteed non-negative. The test runs with ProptestConfig::with_cases(200) to increase the sample size beyond proptest’s default 256 for higher confidence.

Crate	Property Tested	Strategy Type	Cases
`portal-launcher`	ActionId parse/reject, search score finiteness, result ordering	String regex + composite	Default (256)
`portal-spatial`	ZoneId roundtrip, IPC message serialization, homography stability	Integer range + prop_map	Default (256)
`portal-input`	Velocity non-negative, evdev parser panic-free, tracker reset	Float ranges + collections	200 per test
`portal-common`	`assert_bytes_eq` identity and length rejection	`Vec<u8>`	Default (256)
`portal-npu-runtime`	`page_round` alignment invariant	Integer ranges	Default (256)
`portal-context`	Size management, workspace model invariants	Custom strategies	Default (256)
`portal-shell`	UI state transitions	Enum strategies	Default (256)
`portal-llm`	Tokenization and prompt construction	String regex	Default (256)

Sources: portal/launcher/tests/property_tests.rs#L44-L188, portal/input/tests/proptest_fusion.rs#L153-L199, portal/spatial/tests/property_tests.rs#L19-L68, portal/common/tests/property_tests.rs#L8-L25

Fuzzing: Adversarial Input Resilience

Fuzzing protects the parsing surfaces where untrusted or semi-trusted data enters the system. Portal deploys 8 libfuzzer harnesses across 4 fuzz packages, each targeting a specific parser that could crash the daemon on malformed input — a denial-of-service vector in an embedded AR glasses platform. All harnesses use libfuzzer-sys with the #![no_main] attribute and the fuzz_target! macro, built via cargo +nightly fuzz.

The workspace-level fuzz/Cargo.toml#L1-L55 package contains five targets covering the most security-critical parsers. Each target has a curated corpus directory with dozens of seed inputs (both valid and adversarial) that guide the fuzzer toward interesting code paths:

graph LR
    subgraph "Workspace fuzz/"
        RTP["rtp_depacketizer<br/>34 corpus seeds"]
        IPC["ipc_framing<br/>PCP length-prefix + postcard"]
        DESK["desktop_parser<br/>33 corpus seeds"]
        TOML["toml_config<br/>35 corpus seeds"]
        GST["gstreamer_argv<br/>40 corpus seeds"]
    end

    subgraph "Per-crate fuzz packages"
        SPATIAL["portal/spatial/fuzz<br/>IpcMessage::from_bytes"]
        PCP["portal/pcp/stream/fuzz<br/>decode_message"]
        EVDEV["portal/input/fuzz<br/>SlotTracker::process_event<br/>25 corpus seeds"]
    end

    RTP -->|"RtpHeader::parse"| STREAM[portal-stream]
    IPC -->|"decode_message"| PCP_STREAM[pcp-stream]
    DESK -->|"parse_desktop_file"| PCP_NATIVE[pcp-native]
    TOML -->|"toml::from_str"| EXTERNAL["toml crate"]
    GST -->|"shlex::split"| EXTERNAL2["shlex crate"]
    SPATIAL -->|"IpcMessage::from_bytes"| SPATIAL_CRATE[portal-spatial]
    PCP -->|"decode_message"| PCP_STREAM
    EVDEV -->|"process_event"| INPUT[portal-input]

The RTP depacketizer fuzz target illustrates the minimalist harness pattern: it calls RtpHeader::parse on arbitrary &[u8] input, requiring that the parser never panics regardless of input — including truncated buffers, oversized headers, and adversarial bit patterns in version/padding/CSRC fields. The IPC framing target was created as a regression for P1 issue #29, where an oversized payload_len caused a usize subtraction underflow. The .desktop file parser target is notable for its complexity: it writes fuzz input to a temporary file before parsing, exercising the file-I/O code path alongside the line-by-line parser.

A distinctive pattern in Portal is the proptest-fuzz hybrid for the evdev parser. The crate maintains both a cargo-fuzz harness (in portal/input/fuzz/fuzz_targets/fuzz_evdev.rs, requiring nightly) and a proptest sibling (in portal/input/tests/fuzz_evdev.rs#L52-L76) that runs under cargo test on stable Rust. The proptest version feeds arbitrary RawEvdevEvent sequences into SlotTracker::process_event and verifies the parser never panics — this covers the malicious-firmware threat model where a compromised touchscreen feeds crafted event sequences, and it runs on every CI push without the nightly toolchain overhead.

Fuzz Target	Parser Under Test	Input Surface	Threat Model
`rtp_depacketizer`	`RtpHeader::parse`	RTP wire packets from network	Adversarial streaming source
`ipc_framing`	`decode_message`	PCP IPC socket frames	Untrusted local process
`desktop_parser`	`parse_desktop_file`	`.desktop` file contents	Malicious app package
`toml_config`	`toml::from_str`	Daemon configuration files	Tampered config injection
`gstreamer_argv`	`shlex::split`	Pipeline strings, `Exec=` lines	Shell injection via `.desktop`
`fuzz_evdev` (input)	`SlotTracker::process_event`	evdev event stream	Compromised touch firmware
`ipc_framing` (spatial)	`IpcMessage::from_bytes`	Postcard IPC binary	Wayfire plugin crash
`framing` (pcp-stream)	`decode_message`	PCP stream frames	Duplicate of workspace IPC

Sources: fuzz/fuzz_targets/rtp_depacketizer.rs#L1-L22, fuzz/fuzz_targets/ipc_framing.rs#L1-L22, fuzz/fuzz_targets/desktop_parser.rs#L1-L45, portal/input/tests/fuzz_evdev.rs#L52-L76, fuzz/Cargo.toml#L1-L53

Benchmarks: Criterion on Hot Paths

Portal uses Criterion 0.5 with HTML reports to benchmark three latency-critical subsystems. The dedicated benches/Cargo.toml#L1-L34 workspace member packages benchmarks for RTP packetization, spatial IPC, and voice TTS — all paths that must meet sub-millisecond budgets on the Snapdragon X Elite. Additionally, several crates maintain their own per-crate benchmarks (voice, launcher, context, style, and three PCP sub-crates).

The RTP benchmark suite (benches/benches/rtp_encode.rs#L1-L107) measures six operations across varying payload sizes, from header serialization (12 bytes, no allocation) through full-pipeline audio frame processing (960-sample PCM → build → serialize → parse). The benchmark design captures realistic data shapes: 20ms and 10ms audio frames at 48 kHz, H.265 NAL units at 512 bytes and 64 KB. The black_box pattern prevents the compiler from optimizing away the computation:

c.bench_function("rtp/build_and_serialize_h265_large", |b| {
    b.iter(|| {
        let payload = vec![0u8; H265_NAL_LARGE];
        let packet = builder.build_frame(payload, 3600, false);
        let bytes = packet.to_bytes();
        black_box(bytes);
    })
});

The spatial IPC benchmark (benches/benches/spatial_ipc.rs#L96-L133) is particularly noteworthy because it measures real Unix socket roundtrip latency — not just serialization speed. Using criterion::BatchSize::PerIteration, each benchmark iteration creates a fresh UnixStream::pair(), sends an IpcRequest::AssignZone message, reads it server-side, and returns an IpcResponse::Ok. This captures the kernel-level context switch and buffer-copy overhead that the actual compositor daemon experiences. The voice TTS benchmark (benches/benches/voice_tts.rs#L1-L142) measures SSML parsing at three complexity levels (plain text, simple <speak>, and nested prosody/break/emphasis markup), PCM resampling at different frame sizes, and f32-to-s16le sample conversion.

Benchmark Crate	Operations Measured	Data Sizes	Target Budget
`rtp_encode`	Header serialize/parse, audio build, H.265 build+serialize, full pipeline	12 B header, 960–1920 B audio, 512 B–64 KB NAL	Sub-ms per packet
`spatial_ipc`	Request encode, response decode, header roundtrip, postcard serialize, Unix socket roundtrip	Zone assign, homography matrix (9 × f32)	Sub-ms IPC
`voice_tts`	Tone generation, SSML parsing, PCM resampling, f32→s16le conversion	16/22.05/48 kHz, 20–100 ms frames	Real-time audio budget

Sources: benches/benches/rtp_encode.rs#L63-L96, benches/benches/spatial_ipc.rs#L96-L133, benches/benches/voice_tts.rs#L27-L61, benches/Cargo.toml#L1-L34

Miri: Undefined Behaviour Detection

For the FFI-heavy crates that interface with C libraries, kernel ioctls, and DSP FastRPC, Portal runs Miri (the Rust UB detector) in CI against unit-test targets. The .github/workflows/miri.yml#L1-L51 workflow uses nightly Rust with strict provenance checking, symbolic alignment verification, and isolation disabled. The baseline results, established on 2026-07-23, document clean passes: portal-npu-runtime (13 passed, 0 failed), with the spatial and input crates noted as requiring Linux-specific dependencies not available on CI runners.

Miri does not understand kernel ioctls or DSP FastRPC calls, so integration tests are excluded — only --lib unit tests run. The ignored tests at each skip site are documented: FFI type-confusion tests in spatial (intentionally construct invalid pointers), performance budget tests in input (Miri is ~100× slower), and font rasterization tests (fontdue interpreted execution hangs under Miri).

Sources: .github/workflows/miri.yml#L1-L51

CI Integration and Coverage Gate

The test pipeline is split across three workflow files. The .github/workflows/test.yml#L1-L26 workflow runs on every push and pull request with three stages: cargo test --lib --workspace --all-features (unit tests), cargo test --test '*' --workspace --all-features (integration tests), and cargo test --doc --workspace (documentation tests, still continue-on-error: true pending intra-doc link fixes). The .github/workflows/check.yml#L60-L75 workflow handles code coverage via cargo-llvm-cov with an 85% line-coverage floor. This floor accounts for the ~11% gap caused by QNN/ONNX FFI crates whose C-language calls defeat llvm-cov instrumentation — the gap is documented in the v0.2.1 hardening report.

For on-device validation, the scripts/hp_smoke_test.sh#L1-L50 script builds every production binary natively on the Snapdragon X Elite and verifies each daemon can start, answer its IPC handshake, and exit cleanly. This catches target-specific link errors, dynamic library resolution failures, and kernel ABI mismatches that cross-compilation tests on CI x86_64 runners cannot detect.

flowchart TD
    PUSH["Push / Pull Request"] --> FMT["cargo fmt --check"]
    PUSH --> CLIPPY["clippy --lib --bins<br/>strict production gate"]
    PUSH --> CLIPPY_T["clippy --tests<br/>permissive gate"]
    PUSH --> TEST["cargo test --lib + --test<br/>--workspace --all-features"]
    PUSH --> MIRI_RUN["Miri UB scan<br/>npu-runtime --lib"]
    PUSH --> COV["llvm-cov --fail-under-lines 85"]
    TEST --> SYNC["sync-to-hp.sh"]
    SYNC --> SMOKE["hp_smoke_test.sh<br/>on-device daemon lifecycle"]

Sources: .github/workflows/test.yml#L16-L20, .github/workflows/check.yml#L69-L75, scripts/hp_smoke_test.sh#L14-L27, justfile#L53-L57

Running Tests Locally

Developers can run the full test suite via just test, which executes cargo test --workspace --all-features. Individual fuzz targets require the nightly toolchain: cargo +nightly fuzz -p portal-fuzz --target rtp_depacketizer -- -max_total_time=60. The proptest-based fuzz siblings (like fuzz_evdev) run under standard cargo test: cargo test -p portal-input --test fuzz_evdev, or with more cases via PROPTEST_CASES=4096 cargo test -p portal-input --test fuzz_evdev -- --nocapture. Benchmarks are run via cargo bench -p portal-benches and produce HTML reports in target/criterion/.

Sources: justfile#L52-L57, fuzz/fuzz_targets/rtp_depacketizer.rs#L11-L14, portal/input/tests/fuzz_evdev.rs#L13-L17

Next Steps

With the testing strategy understood, the natural progression is to examine how these tests are orchestrated in CI and how security validation integrates into the pipeline:

CI/CD Pipeline: Lint Enforcement, Cross-Compilation, and Release Signing — how the test, lint, coverage, and Miri workflows compose into the full CI gate.
Security Hardening: AppArmor, Stream Authentication, and Supply Chain Audits — how fuzzing targets map to specific security threats and how the .gitleaks.toml and Semgrep rules complement runtime testing.
Latency Budget: Per-Stage Budgets and Benchmark Methodology — how Criterion benchmarks connect to the system-wide latency budget allocation.