NPU Runtime: Safe Rust Wrappers for Qualcomm Hexagon FastRPC

The portal-npu-runtime crate provides a memory-safe, RAII-managed Rust abstraction over the Qualcomm Hexagon CDSP FastRPC kernel interface on the Snapdragon X Elite platform. It is the foundational layer through which every NPU consumer in the Portal stack — voice VAD/STT/TTS, on-device LLM inference, and any future DSP-accelerated workload — allocates DMA-BUF memory, loads the unsigned FastRPC shell, maps buffers into the DSP address space, and dispatches remote procedure calls to the Hexagon DSP. The crate replaces the raw C open/ioctl/mmap ceremony with Rust’s ownership types (OwnedFd, NonNull, BorrowedFd), compile-time thread-safety contracts (Send/Sync), and runtime fences — all while preserving the exact kernel ABI that the FastRPC driver expects.

Sources: portal/npu-runtime/src/lib.rs#L1-L78, portal/npu-runtime/README.md#L1-L21, portal/npu-runtime/Cargo.toml#L1-L31

Architectural Overview

The crate sits between the kernel FastRPC driver (/dev/fastrpc-cdsp-secure) and higher-level consumers. Its central design principle is that every kernel interaction is serialized through a mutex, every file descriptor is owned by an RAII guard, and every unsafe operation carries a # Safety doc section enforced by #![deny(clippy::missing_safety_doc)]. The diagram below shows how the layers connect:

flowchart TB
    subgraph Consumers["NPU Consumers"]
        VAD["Voice VAD/STT"]
        TTS["TTS (Elara VITS)"]
        LLM["LLM Daemon (GenieX)"]
    end

    subgraph Crate["portal-npu-runtime (this crate)"]
        ND["NpuDevice<br/>(Arc&lt;FastRpcSession&gt;)"]
        DMB["DmaBuffer<br/>(OwnedFd + UnsafeCell&lt;ptr&gt;)"]
        FRS["FastRpcSession<br/>(meta_fd + rpc_fd<br/>ioctl_lock: Mutex)"]
        SHV["shell_verify<br/>(SHA-256 + Ed25519)"]
        FFI["fastrpc_ffi<br/>(C structs + ioctl numbers)"]
    end

    subgraph Kernel["Linux Kernel / Hexagon CDSP"]
        DRV["fastrpc driver<br/>/dev/fastrpc-cdsp-secure"]
        DSP["Hexagon DSP<br/>(V73 arch)"]
    end

    VAD --> ND
    TTS --> ND
    LLM --> ND

    ND -->|alloc_dma| FRS
    ND --> DMB
    DMB -->|map_to_dsp / with_data| FRS
    FRS -->|verify_shell_binary| SHV
    FRS -->|libc::ioctl| FFI
    FFI -->|FASTRPC_IOCTL_*| DRV
    DRV <-->|DMA-BUF + RPC| DSP

The crate exposes four public types — NpuDevice, FastRpcSession, DmaBuffer, and the error enum NpuError — plus the raw FFI bindings in fastrpc_ffi. The module boundary is deliberately organized so that the kernel-interface FFI code stays under a 250-LOC review ceiling: shell_verify.rs was extracted from lib.rs precisely to keep the session/FFI code within that limit.

Sources: portal/npu-runtime/src/lib.rs#L34-L51, portal/npu-runtime/src/shell_verify.rs#L1-L10

Type System and Error Model

The error surface is a single thiserror-derived enum that maps every failure mode in the FastRPC lifecycle to a distinct, human-readable variant. The NpuResult<T> alias propagates through all fallible APIs, enabling clean ? chains from the deepest ioctl failure up to the consumer daemon’s error logger.

Error Variant	Trigger	Display Contains
FastRpc(i32)	Raw kernel ioctl returns non-zero rc	The rc integer
OpenFailed { uri, code }	Device open fails for a specific URI	URI + code
InvokeFailed { code }	FastRPC invoke ioctl fails	The code
MmapFailed { code }	FastRPC MMAP ioctl fails	The code
DmaBufAlloc(String)	ALLOC_DMA_BUFF ioctl or mmap failure	Context message
DmaSizeOverflow(usize)	Page-rounding arithmetic overflow	The offending size
DmaExceedsMaxSize(usize)	Size > 256 MB ceiling	Size + “256 MB”
Qnn(String)	QNN model-level failures	QNN error text
ModelNotLoaded	Operation on unloaded model	—
InvalidTensorIndex(u32)	Bad tensor index	The index
SessionBusy	Concurrent inference on single-session DSP	—
ShellVerify(String)	SHA-256 or Ed25519 check fails	Reason text
InvalidFd(RawFd)	Negative or invalid fd	The fd value

Every variant’s Display implementation is pinned by unit tests that assert non-empty output and specific substrings — the wording is part of the operator runbook contract, and silent rewording would cost time during outages.

Sources: portal/npu-runtime/src/error.rs#L1-L53, portal/npu-runtime/src/tests.rs#L776-L854

FastRpcSession: The Core Session Wrapper

FastRpcSession is the single type that wraps the raw FastRPC file descriptors and serializes all kernel ioctls. It holds two OwnedFds — meta_fd for DMA allocation ioctls and rpc_fd for RPC/map operations — matching the discovery that the QAIRT library uses two separate file descriptors rather than one. This is a hardware-specific requirement of the secure device path on the Snapdragon X Elite: opening /dev/fastrpc-cdsp-secure with O_RDONLY | O_NONBLOCK triggers a different kernel code path than O_RDWR, and the session must skip INIT_ATTACH entirely, going directly to INIT_CREATE after verifying the connection.

The critical fields are:

pub struct FastRpcSession {
    meta_fd: OwnedFd,        // for ALLOC_DMA_BUFF / FREE_DMA_BUFF
    rpc_fd: OwnedFd,         // for INIT_CREATE / MMAP / MUNMAP / INVOKE
    domain: i32,             // 3 = CDSP_SECURE_DEVICE
    shell_loaded: Mutex<bool>,
    ioctl_lock: Mutex<()>,   // P0 #5: serialize ioctls per-fd
}

Construction follows a strict three-phase protocol: (1) open both descriptors with O_NONBLOCK, (2) call FASTRPC_IOCTL_GET_DSP_INFO to verify the DSP is alive, (3) verify and load the DSP shell binary via INIT_CREATE. Phases 2 and 3 are each guarded by the ioctl_lock mutex, and phase 3 is guarded by shell_loaded (idempotent — subsequent sessions skip the load).

Sources: portal/npu-runtime/src/lib.rs#L71-L115, portal/npu-runtime/NPU_UNLOCKED.md#L86-L132

The Shell Loading Sequence

The shell-load path is the most safety-critical sequence in the crate. It demonstrates five distinct hardening patterns in sequence:

1. Fail-closed verification — Before reading the shell binary, verify_shell_binary() checks both a SHA-256 hash file at /etc/portal/npu-shell.sha256 and an Ed25519 signature against a hardcoded public key. Any failure returns NpuError::ShellVerify and prevents the shell from being loaded onto the DSP.

2. Bounds-checked allocation — The shell size is checked against MAX_DMA_ALLOC (256 MB) and page-rounded with overflow detection before the allocation ioctl is issued.

3. DMA-BUF allocation + mmap — FASTRPC_IOCTL_ALLOC_DMA_BUFF returns a kernel-managed fd, which is immediately wrapped in OwnedFd. The fd is then mmap’d with PROT_READ | PROT_WRITE | MAP_SHARED for concurrent host/DSP access.

4. Non-overlapping copy — The shell bytes are copied from Rust-owned heap memory into the DMA mapping via copy_nonoverlapping.

5. Error-path teardown ordering — If INIT_CREATE returns non-zero, the host mmap is explicitly munmap’d and the fd is dropped before propagating the error. On success, the fd is released to the kernel (which holds its own reference) but the host mmap is intentionally kept alive for the session’s lifetime to prevent a dangling-pointer use-after-free.

Sources: portal/npu-runtime/src/lib.rs#L146-L281, portal/npu-runtime/src/shell_verify.rs#L35-L73

DMA-BUF Allocation and Free

The alloc_dma method is pub(crate) — all external allocation goes through the safe DmaBuffer constructor on NpuDevice. The method enforces the check order: size > MAX_DMA_ALLOC → DmaExceedsMaxSize; page-round overflow → DmaSizeOverflow; then the ioctl. The free_dma method is pub unsafe fn with a detailed safety contract documenting the required teardown order: (1) FREE_DMA_BUFF ioctl with rc check, (2) munmap, (3) close fd via OwnedFd::drop. This order prevents a kernel double-free if the ioctl fails.

Sources: portal/npu-runtime/src/lib.rs#L290-L411

DmaBuffer: Safe DMA Region Access

DmaBuffer is the safe wrapper that owns a DMA-BUF allocation’s complete lifecycle. It holds the OwnedFd, the host virtual address via UnsafeCell<*mut [u8]> (the UnsafeCell marks interior mutability because the DSP may write concurrently via DMA), an optional DSP address, and an optional Arc<FastRpcSession> for cleanup.

The type is intentionally Send but not Sync. This is a deliberate design decision documented in the source: concurrent shared access to a DMA region without explicit fence ordering has no useful semantics. The with_data and with_data_mut methods are the only sanctioned access patterns — each issues a SeqCst fence before reading or writing to ensure the CPU observes in-flight DSP DMA writes:

pub fn with_data<F: FnOnce(&[u8]) -> R, R>(&self, f: F) -> R {
    fence(Ordering::SeqCst);
    let slice: &[u8] = unsafe { &*self.data_ptr() };
    f(slice)
}

The Drop implementation follows the same ordered teardown as free_dma: if the buffer was mapped to the DSP, it calls unmap_from_dsp (logging on error, not propagating), then calls the session’s free_dma. For untracked buffers (no session Arc), it falls back to a bare munmap since there is no session to issue the ioctl through.

Sources: portal/npu-runtime/src/device.rs#L18-L63, portal/npu-runtime/src/device.rs#L165-L220, portal/npu-runtime/src/device.rs#L223-L244

NpuDevice: Public Device Handle

NpuDevice is the public entry point for NPU consumers. It wraps an Arc<FastRpcSession>, making it cheaply cloneable for sharing across worker threads. The Send + Sync impls are justified at the type level: the Arc<FastRpcSession> is Sync because FastRpcSession’s ioctls are serialized by ioctl_lock, and Arc is Send when its contents are Send + Sync.

The primary method is alloc_dma(size), which delegates to the session and wraps the result in a DmaBuffer that retains an Arc clone of the session for cleanup. Two additional constructors on DmaBuffer — alloc (untracked, caller-managed session) and alloc_tracked (session-owned Arc) — provide flexibility for consumers that manage their own session lifecycle.

Method	Returns	Lifetime Guarantee
NpuDevice::open()	NpuResult<NpuDevice>	Opens CDSP domain 3
NpuDevice::alloc_dma(size)	NpuResult<DmaBuffer>	Buffer freed on DmaBuffer::drop
DmaBuffer::alloc(session, size)	NpuResult<DmaBuffer>	Untracked — caller must free
DmaBuffer::alloc_tracked(session, size)	NpuResult<DmaBuffer>	Session Arc retained
DmaBuffer::map_to_dsp(session)	NpuResult<u64>	Idempotent — caches DSP addr
DmaBuffer::with_data(f)	R	SeqCst fence before read
DmaBuffer::with_data_mut(f)	R	SeqCst fence before write

Sources: portal/npu-runtime/src/device.rs#L14-L63, portal/npu-runtime/src/device.rs#L110-L183

FFI Bindings and ioctl Number Encoding

The fastrpc_ffi module provides raw #[repr(C)] struct definitions for every FastRPC kernel interface struct, plus computed ioctl numbers. The ioctl encoding follows the Linux _IOWR(type, nr, sizeof(struct)) convention with type byte 'R' (0x52). Two const fn helpers handle the encoding:

Ioctl	nr	Struct	Direction
INIT_ATTACH	4	— (simple)	_IOW (1<<30)
INIT_CREATE	5	fastrpc_init_create	_IOWR (3<<30)
ALLOC_DMA_BUFF	1	fastrpc_alloc_dma_buf	_IOWR
FREE_DMA_BUFF	2	u32	_IOWR
INVOKE	3	fastrpc_invoke	_IOWR
MMAP	6	fastrpc_req_mmap	_IOWR
MUNMAP	7	fastrpc_req_munmap	_IOWR
MEM_MAP	10	fastrpc_mem_map	_IOWR
MEM_UNMAP	11	fastrpc_mem_unmap	_IOWR
GET_DSP_INFO	13	fastrpc_ioctl_capability	_IOWR

Unit tests verify that all ioctl numbers are distinct, that the direction/type/size bits match the kernel ABI, and that the size field in each encoded number equals size_of of the corresponding struct. These are kernel ABI invariants — if either the Rust struct or the kernel struct drifts, the ioctl dispatcher silently rejects the call.

The module also provides remote_scalars_make(method, num_in, num_out) which packs a 32-bit scalar descriptor used by the DSP dispatcher: bits 24–32 for method, 12–24 for input argument count, 0–12 for output argument count.

Sources: portal/npu-runtime/src/fastrpc_ffi.rs#L1-L153, portal/npu-runtime/src/tests.rs#L600-L766

Shell Binary Verification: Defense in Depth

Before any shell bytes reach the DSP, verify_shell_binary performs two independent cryptographic checks. The shell binary at /usr/lib/dsp/cdsp/fastrpc_shell_unsigned_3 is the unsigned FastRPC shell that establishes the DSP-side RPC infrastructure. Because it runs in an unsigned protection domain, the host-side crate compensates with host-level verification:

1. SHA-256 hash match — The binary’s hash is compared against a root-owned reference at /etc/portal/npu-shell.sha256. This catches filesystem corruption or tampering by a non-root attacker.

2. Ed25519 signature verification — A .sig sidecar file is verified against a hardcoded Ed25519 public key (the Demain release signing key, held in a 1Password vault). This provides non-repudiation and protects against root-level binary substitution with an unsigned malicious shell.

Both checks must pass; any failure returns NpuError::ShellVerify and prevents INIT_CREATE from being issued. The verification is extracted into shell_verify.rs to keep lib.rs under its review ceiling, and the public verify_shell_binary function is re-exported from the crate root for use by deployment tooling.

Sources: portal/npu-runtime/src/shell_verify.rs#L1-L73, portal/npu-runtime/src/lib.rs#L45

DSP-Side Skeleton: The IDL Contract

The crate includes the FastRPC IDL definition at idl/portal_qnn.idl that defines the remote procedure interface for QNN model operations on the DSP. This IDL is compiled by the Qualcomm qaic compiler into stub (host-side) and skeleton (DSP-side) C code. The skeleton implementation in skel/portal_qnn_skel.c runs on the Hexagon DSP and bridges FastRPC calls to the QNN runtime.

IDL Method	Purpose	DSP-Side Action
model_load	Load a QNN model .so	dlopen + QnnModel_composeGraphs
set_input	Set tensor input data	QnnGraph_setTensor
execute	Run graph inference	QnnGraph_execute
get_output	Read tensor output	QnnGraph_getTensor + memcpy
model_unload	Free model resources	QnnContext_free + dlclose

The skeleton maintains a fixed-size model slot table (MAX_MODELS = 4, MAX_GRAPHS_PER_MODEL = 8) and supports two model loading strategies: dlopen of a model .so file that exports QnnModel_composeGraphs, or a direct function pointer for pre-loaded models. The build pipeline uses a Docker container with the Hexagon LLVM toolchain (hexagon-clang + qaic) to cross-compile these skeletons into shared libraries targeting the V75 DSP architecture.

An echo.idl provides a minimal test interface (echo_test_echo) for verifying the FastRPC round-trip path without QNN model dependencies.

Sources: portal/npu-runtime/idl/portal_qnn.idl#L1-L42, portal/npu-runtime/idl/echo.idl#L1-L6, portal/npu-runtime/skel/portal_qnn_skel.c#L1-L284, portal/npu-runtime/docker/build_skel.sh#L1-L53

Hardware Configuration and Compatibility

The runtime is validated on the HP EliteBook Ultra G1q (Snapdragon X Elite / X1E80100) with a specific firmware/toolchain combination discovered through extensive experimentation. The working configuration is:

Component	Version	Source
CDSP Firmware	CDSP.HT.2.9.c1-00046-HAMOA-1	HP SoftPaq sp162865.exe
QAIRT SDK	v2.44.0.260225	Qualcomm Software Center
DSP Skel Architecture	V73 (not V75)	DSP self-reports as V73
FastRPC Shell	fastrpc_shell_unsigned_3	HP driver pack

The c1-00046 firmware is critical: it is HP-signed (accepted by TrustZone PIL auth) and supports unsigned modules. Earlier c2-00051 firmware was TrustZone-locked and rejected unsigned modules with error 0x80000A1A. The c1-00069 IoT EVK firmware was rejected by TrustZone entirely because it was not HP-signed. Runtime PM must be disabled for the CDSP to prevent a known crash at sleep_statsi.c:537 approximately 90 seconds after boot.

The environment is configured via config/portal-npu.sh, which sets LD_LIBRARY_PATH to the QAIRT v2.44 directory and ADSP_LIBRARY_PATH to /usr/lib/dsp/cdsp. Deployment is handled by scripts/deploy_npu.sh, which restores firmware, DSP libraries, and QAIRT runtime from a snapshot tarball.

Sources: portal/npu-runtime/NPU_WORKING.md#L1-L70, portal/npu-runtime/NPU_UNLOCKED.md#L20-L132, portal/npu-runtime/config/portal-npu.sh#L1-L6, portal/npu-runtime/scripts/deploy_npu.sh#L1-L88

Thread Safety Contracts

The crate’s concurrency model is pinned by compile-time tests that assert the Send/Sync bounds. These are not auto-derived — they are unsafe impl with detailed safety justifications, because the types contain raw pointers and UnsafeCell:

Type	Send	Sync	Justification
FastRpcSession	✅	✅	All ioctls serialized by ioctl_lock; kernel accepts calls from any thread
NpuDevice	✅	✅	Holds only Arc<FastRpcSession>
DmaBuffer	✅	❌	!Sync intentional — concurrent access needs explicit fences

If a future field addition introduces Rc, RefCell, or other !Send/!Sync types, the compile-time assertions in tests.rs will fail to compile, immediately surfacing the regression.

Sources: portal/npu-runtime/src/lib.rs#L498-L508, portal/npu-runtime/src/device.rs#L223-L244, portal/npu-runtime/src/tests.rs#L230-L263

Testing Strategy

The crate employs a three-tier testing approach:

Tier 1 — Inline unit tests (src/tests.rs, 912 lines) exercise private helpers (page_round, MAX_DMA_ALLOC) and pub(crate) rejection paths using /dev/null-backed sessions that never issue real ioctls. These run on every platform including macOS CI. Property tests (proptest) verify that page-rounding always yields a page multiple, never decreases, and adds less than one page of overhead.

Tier 2 — Integration tests (tests/dma_allocation.rs) exercise the public API surface: NpuError Display impls, NpuResult round-tripping, and the FastRpcSession constructor. Hardware-dependent paths are #[ignore]-gated and run on the Spaceboard via cargo test -- --ignored --test-threads=1 (single-threaded execution is mandatory because the CDSP shell-load path is serialized at the kernel level).

Tier 3 — Hardware smoke test (src/bin/npud-test/main.rs) is a standalone binary that opens the device, allocates a 1 MB DMA-BUF, maps it to the DSP, writes a test pattern, reads it back, and verifies integrity. It is run manually on-device to validate the full kernel→DSP→kernel round-trip.

Sources: portal/npu-runtime/src/tests.rs#L1-L120, portal/npu-runtime/tests/dma_allocation.rs#L138-L261, portal/npu-runtime/src/bin/npud-test/main.rs#L1-L73

Consumer Integration

The portal-npu-runtime crate is consumed by the voice pipeline through a trait abstraction layer in portal/voice/src/model_manager/npu_runtime/. The NpuRuntime trait defines a uniform interface (load_model, load_llm, run, unload, is_available, utilization) that the PortalNpuRuntime struct implements. This layer manages model handles, tracks loaded models in a HashMap<usize, LoadedModel> behind a Mutex, and delegates inference to the inference submodule. The trait abstraction allows the voice pipeline to work with or without NPU hardware — when NPU features are not compiled in, all operations return VoiceError::NpuRuntimeInitFailed with a descriptive message.

Sources: portal/voice/src/model_manager/npu_runtime/mod.rs#L1-L21, portal/voice/src/model_manager/npu_runtime/portal_runtime.rs#L48-L70

Next Steps

• NPU Scheduling — Learn how multiple workloads (VAD, TTS, LLM) arbitrate access to the single CDSP session.
• On-Device LLM Daemon — See how the LLM daemon builds on this runtime to run llama.cpp via the Hexagon DSP.
• Voice Pipeline — Understand how the voice crate consumes the NpuRuntime trait for VAD and TTS inference.