ARM64 Sandbox Environment Setup

The Portal platform runs on ARM64 hardware — primarily the HP Spaceboard (Snapdragon X Elite) and legacy Orange Pi 5 Max (RK3588). Development, however, happens on x86_64 workstations. This page documents the cross-compilation toolchain, the QEMU-based Docker sandbox for local ARM64 binary testing, and the build automation that ties them together. By the end, you will understand how to compile Rust and C++ artifacts for ARM64, smoke-test them locally without touching hardware, and integrate the sandbox into CI pipelines.

Sources: run_sandbox.sh#L1-L29, rust-toolchain.toml#L1-L4

---

Architecture: Two Layers of ARM64 Enablement

The sandbox is not a single tool — it is a two-layer system. The first layer compiles native ARM64 binaries on your x86_64 machine using cross-compilers. The second layer executes those binaries locally through QEMU user-mode emulation inside a Docker container that mirrors the target OS. This separation lets you catch logic bugs and ABI mismatches before deploying to physical devices, which is critical when team members share limited hardware.

flowchart TB
    subgraph Host["x86_64 Development Workstation"]
        SRC["Source Code (Rust + C++)"] --> CC["Cross-Compilation Layer"]
        CC --> |aarch64-linux-gnu-gcc| BIN["ARM64 Binaries<br/>target/aarch64-unknown-linux-gnu/release/"]
        BIN --> SBX["QEMU Sandbox Layer"]
        SBX --> |docker run --platform linux/arm64| EMU["QEMU User-Mode Translation<br/>arm64v8/debian:bookworm"]
        EMU --> RESULT{"Pass/Fail"}
        RESULT --> |pass| DEPLOY["Deploy to Device"]
        RESULT --> |fail| SRC
    end

    subgraph CI["GitHub Actions CI"]
        CROSS["cross build<br/>(cross-rs Docker image)"] --> ARTIFACT["Upload ARM64 Artifacts"]
    end

    Host -.-> |push| CI

The flowchart above shows the complete development cycle: source code is cross-compiled for ARM64 on the left, tested in the QEMU sandbox in the center, and either looped back for fixes or deployed to real hardware. The CI pipeline at the bottom performs the same cross-compilation using cross-rs pre-built Docker images.

Sources: run_sandbox.sh#L6-L28, Cross.toml#L1-L13, .github/workflows/cross-compile.yml#L8-L18

---

Layer 1: Cross-Compilation Toolchain

Rust Cross-Compilation

The workspace pins Rust 1.97.0 as the stable toolchain with rustfmt, clippy, and rust-src components. Cargo is configured to use the aarch64-linux-gnu-gcc linker for the aarch64-unknown-linux-gnu target triple. This means cargo build --target aarch64-unknown-linux-gnu produces native ARM64 ELF binaries without any Docker involvement — the GNU cross-compiler runs directly on your host.

Configuration File	Role	Key Setting
rust-toolchain.toml#L1-L4	Toolchain pin	channel = "1.97.0"
.cargo/config.toml#L1-L2	Linker mapping	linker = "aarch64-linux-gnu-gcc"
Cross.toml#L1-L13	cross tool images	ghcr.io/cross-rs/aarch64-unknown-linux-gnu:main
Cargo.toml#L148-L149	Workspace metadata	rust-version = "1.97.0", edition = "2021"

On macOS (the primary dev OS for this project), you cannot install aarch64-linux-gnu-gcc natively. Two strategies apply:

1. Use cross (recommended) — The Cross.toml#L8-L9 maps the aarch64-unknown-linux-gnu target to a pre-built Docker image (ghcr.io/cross-rs/aarch64-unknown-linux-gnu:main). Running cross build --target aarch64-unknown-linux-gnu --release transparently builds inside a container that has the full cross-compiler and ARM64 system libraries. No local cross-compiler installation needed.

2. Use an SSH build host — The project’s HP Spaceboard (accessible via ssh hp) compiles natively on ARM64. The scripts/sync-to-hp.sh#L1-L22 script rsyncs source from the Mac worktree to the Spaceboard, where cargo build --release produces native ARM64 binaries without any cross-compilation overhead.

Sources: rust-toolchain.toml#L1-L4, .cargo/config.toml#L1-L2, Cross.toml#L1-L13, scripts/sync-to-hp.sh#L1-L22

C++ Cross-Compilation

The Wayfire plugins and streaming pipeline are written in C/C++ and use CMake. Two identical toolchain files exist for CMake-based cross-compilation:

File	Purpose
toolchains/aarch64-linux-gnu.cmake#L1-L9	Root-level CMake toolchain for cross-compile
scripts/deploy/aarch64-toolchain.cmake#L1-L14	Deployment-specific copy (used by deploy.sh)

Both set CMAKE_C_COMPILER and CMAKE_CXX_COMPILER to aarch64-linux-gnu-gcc and aarch64-linux-gnu-g++ respectively, and configure CMAKE_FIND_ROOT_PATH_MODE_* so that library and header searches target the ARM64 sysroot while programs are found on the host.

The scripts/deploy/deploy.sh#L12-L16 script demonstrates the full C++ cross-compilation invocation: cmake .. -DCMAKE_TOOLCHAIN_FILE=../aarch64-toolchain.cmake followed by make -j$(nproc).

Sources: toolchains/aarch64-linux-gnu.cmake#L1-L9, scripts/deploy/deploy.sh#L11-L16

Voice Daemon: Special Cross-Compilation Needs

The voice daemon (portal-voiced) requires ALSA system libraries that live in the ARM64 sysroot. The justfile#L37-L48 defines a dedicated build-voiced recipe that sets three PKG_CONFIG environment variables to force pkg-config to resolve against the ARM64 sysroot:

PKG_CONFIG_PATH=/usr/lib/aarch64-linux-gnu/pkgconfig \
PKG_CONFIG_SYSROOT_DIR=/usr/aarch64-linux-gnu \
PKG_CONFIG_ALLOW_CROSS=1 \
cargo build --release --bin portal-voiced \
    --target aarch64-unknown-linux-gnu \
    --no-default-features --features alsa-fallback \
    -p portal-voice

The alsa-fallback feature flag disables PipeWire dependency and uses ALSA directly — necessary because the PipeWire dev headers are often not available in cross-compilation sysroots.

Sources: justfile#L37-L48, portal/voice/Cargo.toml#L72-L103

---

Layer 2: QEMU Docker Sandbox

How the Sandbox Works

The run_sandbox.sh#L1-L29 script is the primary entry point for local ARM64 binary testing. It launches a Docker container from the arm64v8/debian:bookworm image with the --platform linux/arm64 flag, which instructs Docker to use QEMU user-mode emulation for transparent ARM64 instruction translation on x86_64 hosts.

The container mirrors the target hardware’s userspace environment. This is critical because binary compatibility depends on matching the C library (glibc) and C++ runtime (libstdc++) versions — a binary compiled against glibc 2.36 will fail to run against glibc 2.31, even on the correct architecture.

Component	Sandbox Container	Orange Pi 5 Max	HP Spaceboard	Match
Base image	arm64v8/debian:bookworm	Debian 12 Bookworm	Debian testing (trixie)	✅ / ⚠️
Architecture	aarch64 (via QEMU)	aarch64	aarch64	✅
glibc	2.36	2.36	2.40+	✅ / ⚠️
Kernel	Host kernel (x86_64)	5.10.160-rockchip-rk3588	7.0.13-iris	❌ (userspace only)

The sandbox matches userspace ABI. Kernel-level interfaces (ioctls, DRM, NPU FastRPC) are not emulated — those require real hardware or hardware-specific mocking.

Sources: run_sandbox.sh#L6-L28, docs/ARM64_Sandbox_Environment_Setup.md#L60-L79

Environment Variables

The sandbox injects two environment variables that your code can check:

Variable	Value	Purpose
IS_SANDBOX	1	Signals to application code that hardware is emulated
TARGET_ARCH	arm64	Identifies the target architecture for conditional logic

Sources: run_sandbox.sh#L25-L26

---

Setup and Usage

One-Time Prerequisites

flowchart LR
    A["1. Install Docker"] --> B["2. Install QEMU + binfmt"]
    B --> C["3. Register QEMU<br/>with kernel"]
    C --> D["4. Verify emulation"]
    D --> E["Ready!"]

    C -.-> |"docker run --rm --privileged<br/>multiarch/qemu-user-static<br/>--reset -p yes"| F["binfmt_misc<br/>registered"]

On Linux hosts, you need QEMU user-static and binfmt-support registered with the kernel:

# Install QEMU and binfmt-support
sudo apt-get update
sudo apt-get install -y qemu-user-static binfmt-support

# Register QEMU for all architectures (one-time, survives reboot on most systems)
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

# Verify ARM64 emulation works
docker run --rm --platform linux/arm64 arm64v8/debian:bookworm uname -m
# Expected: aarch64

On macOS with Apple Silicon, Docker Desktop runs Linux containers via a VM, and ARM64 containers execute natively (no QEMU needed). On macOS with Intel, Docker Desktop includes QEMU emulation but performance is significantly slower.

Sources: docs/ARM64_Sandbox_Environment_Setup.md#L86-L108

Running Binaries in the Sandbox

The run_sandbox.sh script accepts any command as its argument and executes it inside the ARM64 container:

# Cross-compile a binary for ARM64
cargo build --target aarch64-unknown-linux-gnu --release -p portal-wm

# Test it in the sandbox
./run_sandbox.sh ./target/aarch64-unknown-linux-gnu/release/portal-wm --help

# Run Python validation scripts
./run_sandbox.sh python3 scripts/validate_config.py

# Interactive shell
./run_sandbox.sh bash

The script mounts the current working directory read-write at /workspace inside the container, so file I/O is shared. It uses --network host so network-dependent tests work without additional Docker network configuration.

Sources: run_sandbox.sh#L20-L28, docs/ARM64_Sandbox_Environment_Setup.md#L143-L161

Build Automation with Justfile

The justfile#L1-L67 provides convenience recipes for common build operations:

Recipe	Description	Target
just build-rust	Build all Rust workspace crates for ARM64	aarch64-unknown-linux-gnu --release
just build-crate <name>	Build a specific crate for ARM64	aarch64-unknown-linux-gnu --release
just build-cpp	Build Wayfire C++ plugins (native CMake)	Build host arch
just build-voiced	Build voice daemon with ALSA fallback + cross PKG_CONFIG	aarch64-unknown-linux-gnu
just build-all	Run all four build recipes sequentially	Mixed
just test	Run workspace tests (native)	Host arch
just clean	Remove all build artifacts	All targets

Sources: justfile#L8-L67

---

Hardware Abstraction Strategy

What the Sandbox Cannot Emulate

The QEMU user-mode sandbox translates CPU instructions but has no access to physical hardware peripherals. Any code path that touches device-specific interfaces must be mocked:

Hardware	Sandbox Behavior	Mock Strategy
GPIO pins	Not available	Conditional init via IS_SANDBOX=1
NPU (Hexagon DSP)	Not available	Use CPU stub paths
GPU (Mali/Adreno)	Limited — no EGL/DRM	Skip rendering tests
RKNN (Rockchip NPU)	Not available	Mock inference calls
VPU (Video encoder)	Not available	Mock codec interface
Wayland compositor	Not available	Skip compositor-dependent tests

Sources: docs/ARM64_Sandbox_Environment_Setup.md#L182-L196

Detecting the Sandbox in Code

Application code can branch on the IS_SANDBOX environment variable to select mock implementations. The pattern applies to any language — Rust, Python, or C:

import os

class HardwareManager:
    def __init__(self):
        self.is_sandbox = os.getenv('IS_SANDBOX') == '1'
        if self.is_sandbox:
            self._init_mock_hardware()
        else:
            self._init_real_hardware()

For Rust crates, the same detection pattern uses std::env::var("IS_SANDBOX") at initialization time. The CI pipeline’s Miri job further validates that FFI-heavy crates (spatial, input, voice, npu-runtime) have correct #[cfg(test)] stub paths that execute on x86_64 without requiring actual hardware.

Sources: docs/ARM64_Sandbox_Environment_Setup.md#L198-L225, .github/workflows/miri.yml#L1-L51

---

CI/CD Integration

Cross-Compilation in GitHub Actions

The .github/workflows/cross-compile.yml#L1-L28 workflow runs on every push and pull request. It installs cross (v0.2.5), builds the entire workspace for ARM64, runs tests (tolerating hardware-dependent failures), and uploads artifacts:

build-arm64:
  runs-on: ubuntu-latest
  steps:
    - uses: actions/checkout@v4
    - uses: dtolnay/rust-toolchain@stable
      with: { targets: aarch64-unknown-linux-gnu }
    - run: cargo install cross --git https://github.com/cross-rs/cross --tag v0.2.5
    - run: cross build --target aarch64-unknown-linux-gnu --release --workspace
    - run: cross test --target aarch64-unknown-linux-gnu --workspace --release || true

The || true on the test step acknowledges that some tests require actual ARM64 hardware (NPU, DRM, Wayland) and will fail in CI. This is a deliberate trade-off: the cross-compile build catches link errors and ABI mismatches, while hardware-dependent correctness is verified by the HP smoke test suite.

ARM64 Clippy Gate

The .github/workflows/check.yml#L77-L87 workflow includes a clippy-arm64 job that runs cross clippy --target aarch64-unknown-linux-gnu. It is currently continue-on-error: true because ARM64-specific feature combinations can trigger lints that don’t appear on x86_64.

Release Pipeline

The .github/workflows/release.yml#L1-L24 workflow builds ARM64 release binaries via cross, generates a CycloneDX SBOM, and signs each binary with cosign keyless (Ed25519 via GitHub OIDC). The signed artifacts (binary + .sig + .pem + .bundle) are uploaded as GitHub release assets.

Sources: .github/workflows/cross-compile.yml#L8-L28, .github/workflows/check.yml#L77-L87, .github/workflows/release.yml#L18-L24

---

On-Device Verification: HP Smoke Test Suite

Once binaries pass the sandbox, the project uses two complementary smoke test scripts that run on the actual hardware:

Script	Location	What It Verifies
scripts/hp_smoke_test.sh#L1-L126	Runs from Mac via SSH to hp	Binary linking, --help exit codes, unit tests, cmocka tests
scripts/hp-smoke-suite.sh#L1-L295	Runs on the Spaceboard itself	Service health, UDP port binding, DRM scanout, IPC socket, restart cycles

The hp_smoke_test.sh script builds all production binaries on the Spaceboard, runs Rust unit tests for FFI-heavy crates (portal-npu-runtime, portal-spatial, portal-input, portal-voice), and builds/runs the C streaming cmocka tests. The hp-smoke-suite.sh goes deeper — it probes live systemd services, checks the voice daemon’s UDP socket, verifies the spatial IPC socket responds to QueryStatus, and performs three boot cycles of portal-input to catch DSP/DRM fd leaks.

Sources: scripts/hp_smoke_test.sh#L58-L108, scripts/hp-smoke-suite.sh#L62-L89

---

Troubleshooting

Issue	Root Cause	Solution
exec format error	QEMU not registered with binfmt_misc	docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
Slow execution (~10× overhead)	QEMU instruction translation	Expected — use for smoke tests only, not benchmarks
Missing shared libraries	Container lacks runtime deps	Add apt-get install to the script’s bootstrap line
PKG_CONFIG_ALLOW_CROSS errors	pkg-config refuses cross-compile	Set explicitly: PKG_CONFIG_ALLOW_CROSS=1 (see just build-voiced)
linker aarch64-linux-gnu-gcc not found	Cross-compiler not installed (Linux)	sudo apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu or use cross
Network failures in container	Docker network isolation	Script uses --network host; verify Docker allows host networking

Sources: docs/ARM64_Sandbox_Environment_Setup.md#L259-L275, justfile#L37-L48

Verification Commands

# Verify QEMU is registered for ARM64
ls /proc/sys/fs/binfmt_misc/ | grep qemu

# Confirm ARM64 container emulation
docker run --rm --platform linux/arm64 arm64v8/debian:bookworm uname -m
# Output: aarch64

# Verify sandbox environment variables
./run_sandbox.sh bash -c 'echo "ARCH=$(uname -m) IS_SANDBOX=$IS_SANDBOX"'
# Output: ARCH=aarch64 IS_SANDBOX=1

Sources: docs/ARM64_Sandbox_Environment_Setup.md#L289-L302

---

Development Workflow Summary

flowchart LR
    A["Edit Code<br/>(macOS x86_64)"] --> B{"Build Strategy"}
    B --> |Cross-compile| C["cross build --target<br/>aarch64-unknown-linux-gnu"]
    B --> |SSH build| D["sync-to-hp.sh<br/>cargo build on Spaceboard"]
    C --> E["run_sandbox.sh<br/>(QEMU local test)"]
    E --> F{"Pass?"}
    F --> |Yes| G["Deploy to device"]
    F --> |No| A
    D --> H["hp_smoke_test.sh<br/>(on-device smoke test)"]
    G --> H
    H --> I["hp-smoke-suite.sh<br/>(integration test)"]
    I --> J{"Pass?"}
    J --> |Yes| K["Production ✅"]
    J --> |No| A

The complete cycle moves from local editing through cross-compilation, sandbox testing, on-device smoke tests, and finally integration tests. Each stage catches different classes of bugs — the sandbox catches ABI and logic issues, while the on-device tests catch hardware interface problems.

Sources: run_sandbox.sh#L1-L29, scripts/sync-to-hp.sh#L1-L22, scripts/hp_smoke_test.sh#L58-L79, scripts/hp-smoke-suite.sh#L62-L89

---

Next Steps

Now that you understand the sandbox and cross-compilation environment, the following pages provide deeper context on how the built artifacts are deployed and tested:

• Dual-Display Wayland Architecture — How the compositor binaries you just built interact with the hardware display pipeline
• CI/CD Pipeline — Full CI/CD pipeline details including lint enforcement and release signing
• Systemd Service Management — How the deployed ARM64 binaries are managed as systemd services on the Spaceboard
• Testing Strategy — Comprehensive testing approach including property tests and fuzzing