Spatial Warp Rendering: Homography-Based Head Tracking Correction

The Portal platform renders a 2560×1080 virtual canvas onto AR glasses (INMO AIR3) through a compositor-stage perspective transformation layer. This page documents the dual-implementation architecture: a Rust homography solver crate with GLSL shaders (portal-warp), and the active C++ Wayfire plugin (portal-spatial-warp) that applies trapezoidal perspective correction to views based on their spatial zone assignment. The Rust crate provides the mathematical foundation and rendering primitives; the C++ shim consumes them at runtime through a dlopen-mediated FFI boundary.

Sources: portal/wayfire-plugins/portal-spatial-warp/plugin.cpp#L1-L177, portal/warp/src/lib.rs#L1-L92

---

System Architecture: Two Implementations, One Goal

The warp subsystem is structured as two cooperating layers. The portal-warp Rust cdylib (portal/warp/) contains the DLT homography solver, GLSL ES 3.00 shader source, and a trait-based WarpRenderer abstraction for OpenGL ES 3.0 rendering. The portal-spatial-warp C++ Wayfire plugin (portal/wayfire-plugins/portal-spatial-warp/) is the active production code path — it uses Wayfire’s built-in view_3d_transformer_t pipeline to apply perspective distortion directly in the compositor’s scene graph. The Rust crate’s FFI exports are currently dead code: the C++ shim loads libportal_spatial_plugin.so only for the zone-assignment lookup function (portal_spatial_zone_for_view), not for any warp rendering call. The FFI surface is explicitly marked #[deprecated] and accepted as documented risk pending future removal.

flowchart TB
    subgraph "Wayfire Compositor Process"
        WP["portal-spatial-warp\nC++ Wayfire Plugin"]
        SP["portal-spatial\nC++ Wayfire Plugin"]
        TF["view_3d_transformer_t\n(Wayfire Built-in)"]
    end

    subgraph "libportal_spatial_plugin.so"
        ZFV["portal_spatial_zone_for_view\n(FFI: zone lookup)"]
    end

    subgraph "libportal_warp.so (Dead Code)"
        WI["warp_init / warp_fini"]
        WH["warp_set_homography"]
        WA["warp_apply"]
    end

    subgraph "portal-spatial crate"
        HS["HomographySolver\n(DLT, nalgebra f32)"]
        FFH["spatial_homography_*\nFFI wrappers"]
    end

    WP -->|"dlopen + dlsym"| ZFV
    WP -->|"compute_trapezoid_warp"| TF
    SP -->|"view placement"| TF
    TF -->|"GL render"| GPU["Mali-G610 / Adreno X1-85"]

    WI -.->|"deprecated, unused"| WH
    WH -.-> WA
    HS --> FFH

The C++ plugin communicates with the Rust spatial library through a single FFI function. On initialization, it calls dlopen on libportal_spatial_plugin.so, after verifying a SHA-256 sidecar manifest. It then resolves portal_spatial_zone_for_view via dlsym — the only symbol it actually calls. When a Wayland view is mapped, the plugin queries the zone ID (0=Center, 1=LeftPeripheral, 2=RightPeripheral) and, for peripheral zones, computes a trapezoidal warp matrix using GLM and applies it through Wayfire’s view_3d_transformer_t mechanism.

Sources: portal/wayfire-plugins/portal-spatial-warp/plugin.cpp#L86-L116, portal/warp/src/lib.rs#L1-L27

---

DLT Homography Solver: Mathematical Foundation

The homography computation lives in two parallel implementations — one in portal-warp (f64 precision, used by the dead-code FFI path) and one in portal-spatial (f32 precision, exposed through the spatial crate’s FFI). Both implement the Direct Linear Transform (DLT) algorithm, the standard method for computing a 3×3 projective transformation from four point correspondences.

Algorithm: From Point Pairs to Projection Matrix

Given four source points src[i] and four destination points dst[i], the solver constructs an 8×9 coefficient matrix A where each point correspondence contributes two rows:

Row	Equation
2i	[-x, -y, -1, 0, 0, 0, x·x', y·x', x']
2i+1	[0, 0, 0, -x, -y, -1, x·y', y·y', y']

The homography vector h (the 9-element flattened matrix) is the right null vector of A — the eigenvector of AᵀA corresponding to its smallest eigenvalue. Because a thin SVD on the non-square 8×9 matrix cannot directly expose the null space, the implementation computes the 9×9 symmetric matrix AᵀA and performs a symmetric eigendecomposition instead. The resulting vector is reshaped into a 3×3 matrix and normalized so that h[2][2] = 1.

Sources: portal/warp/src/homography.rs#L51-L158, portal/spatial/src/homography.rs#L103-L178

Degeneracy Detection and Error Handling

The solver enforces three layers of numerical safety before returning a matrix. Collinearity detection computes the 2D cross product for all four combinations of three points; if any triplet is collinear within a 1e-6 tolerance, the DLT system is rank-deficient and the solver returns a typed error rather than a garbage matrix. Normalization failure checks whether |h[2][2]| < 1e-10, which indicates the destination plane has been mapped to the line at infinity. Finiteness validation rejects any result containing NaN or infinity after the SVD, catching numerical instabilities that slip past the earlier gates.

Error Variant	Trigger	Consequence
CollinearSourcePoints	Any 3 of 4 source points collinear (cross product < 1e-6)	DLT system rank-deficient
CollinearDestinationPoints	Any 3 of 4 destination points collinear	Same degeneracy in target
SingularMatrix	Eigendecomposition fails to converge	No null vector found
NormalizationFailure	`	h[2][2]
InvalidHomography	Any entry is NaN or ±∞	Numerical breakdown

Sources: portal/warp/src/error.rs#L24-L61, portal/warp/src/homography.rs#L12-L26

---

The Spatial Crate’s Homography API

The portal-spatial crate provides a higher-level HomographySolver struct with f32 precision (optimized for GPU consumption) that wraps the DLT algorithm and adds utility operations for zone geometry. Beyond the raw 4-point solver, it offers compute_zone_homography which directly constructs a scale-rotate-translate matrix from center position, scale factor, and rotation angle — useful for placing zone content without needing explicit corner correspondences. The apply_homography method performs homogeneous coordinate transformation with a 1e-6 zero-guard on the homogeneous divisor to avoid division by near-zero w.

Sources: portal/spatial/src/homography.rs#L204-L285

This solver is exposed to C/C++ consumers through the spatial crate’s FFI layer, which wraps each call in a panic guard (ffi_boundary_guard) and uses a Tracked<T> smart pointer with atomic magic validation to prevent use-after-free and type confusion. The matrix crosses the ABI boundary as a SpatialMatrix3x3 — a #[repr(C)] struct containing a flat c_float[9] array in row-major order.

Sources: portal/spatial/src/ffi/homography.rs#L17-L140, portal/spatial/src/ffi/types.rs#L58-L81

---

GLSL ES 3.00 Warp Shaders

The vertex and fragment shaders target OpenGL ES 3.0, matching the Mali-G610 GPU on the Orange Pi 5 Max deployment platform. The shaders form a two-stage pipeline: the vertex shader applies the homography to texture coordinates, and the fragment shader performs a simple textured sample at the warped UV position.

Vertex Shader: Homography in Homogeneous Coordinates

#version 300 es
in vec2 a_position;  // Quad vertex position (-1 to 1)
in vec2 a_texcoord;  // Texture coordinate (0 to 1)
uniform mat4 u_homography;  // 3x3 homography as mat4 (row-major)
out vec2 v_texcoord;

void main() {
    gl_Position = vec4(a_position, 0.0, 1.0);
    vec3 hom_uv = vec3(a_texcoord, 1.0);
    vec3 transformed = (u_homography * vec4(hom_uv, 1.0)).xyz;
    v_texcoord = transformed.xy / transformed.z;
}

The key technique is converting texture coordinates to homogeneous form (u, v, 1), applying the 3×3 homography embedded in a mat4 uniform, then performing the perspective divide (transformed.xy / transformed.z) to recover 2D coordinates. This divide is what creates the non-linear foreshortening effect — distant regions of the texture are sampled more densely than near regions, simulating how a flat surface appears when viewed at an angle.

Fragment Shader: Minimal Texture Sampling

The fragment shader is deliberately minimal — all perspective work happens in the vertex stage. The v_texcoord varying is interpolated across the triangle strip, and the GPU’s built-in perspective-correct interpolation handles the rest automatically for the interior of each triangle. The fragment stage simply samples u_texture at the interpolated coordinate with texture(), relying on the GL state’s configured minification/magnification filters for quality.

Sources: portal/warp/src/shader.rs#L17-L74

---

WarpRenderer: OpenGL ES 3.0 Pipeline

The WarpRenderer struct manages the GPU resources for rendering a textured quad with homography transformation. It owns four GL handles — a shader program, a VAO, a VBO, and cached uniform/attribute locations — and exposes a single render_quad method that performs a complete render-to-texture pass.

Full-Screen Quad Geometry

The renderer uploads a four-vertex triangle strip defining a full-screen quad in clip space, interleaved with texture coordinates:

Vertex	Position (x, y)	TexCoord (u, v)	Corner
0	(-1.0, -1.0)	(0.0, 0.0)	Bottom-left
1	(1.0, -1.0)	(1.0, 0.0)	Bottom-right
2	(-1.0, 1.0)	(0.0, 1.0)	Top-left
3	(1.0, 1.0)	(1.0, 1.0)	Top-right

The vertex format is [x, y, u, v] — 4 floats (16 bytes) per vertex, with the a_position attribute at offset 0 and a_texcoord at offset 8. A GL_TRIANGLE_STRIP draw call with count=4 renders the quad as two triangles.

Homography-to-mat4 Packing

The render_quad method converts the [[f64; 3]; 3] Rust matrix into a mat4 uniform by embedding the 3×3 values in the top-left 3×3 block of a 4×4 identity matrix. This allows the GLSL vertex shader to apply the homography as a mat4 * vec4 multiply, with the extra row/column being identity — the perspective terms in the third row (h[2][0], h[2][1]) survive because only transformed.z is read from the result, and the 1.0 in position [3][3] preserves the w component.

let homo_f32: [f32; 16] = [
    h[0][0] as f32, h[0][1] as f32, h[0][2] as f32, 0.0,
    h[1][0] as f32, h[1][1] as f32, h[1][2] as f32, 0.0,
    h[2][0] as f32, h[2][1] as f32, h[2][2] as f32, 0.0,
    0.0,             0.0,             0.0,             1.0,
];

The f64-to-f32 truncation is intentional — the shader pipeline operates at mediump precision (16-bit float), so the extra mantissa bits in f64 would be discarded by the GPU anyway.

Sources: portal/warp/src/renderer.rs#L237-L330, portal/warp/src/renderer.rs#L371-L440

---

Active Production Path: The C++ Wayfire Plugin

While the Rust crate provides the rendering infrastructure, the actual runtime uses Wayfire’s built-in 3D transformer mechanism. The C++ plugin (portal-spatial-warp/plugin.cpp) hooks into Wayfire’s view lifecycle signals and applies perspective distortion through wf::scene::view_3d_transformer_t.

Trapezoidal Warp via GLM

The C++ plugin’s compute_trapezoid_warp method constructs a 4×4 projection matrix that creates a horizontal keystone effect. It sets the proj[2][0] element (the X-perspective term in GLM’s column-major convention) to ±WARP_STRENGTH, where the sign depends on the zone:

Zone	perspective_x	Visual Effect
ZONE_CENTER (0)	N/A (identity)	No distortion
ZONE_LEFT_PERIPHERAL (1)	+0.15	Right edge contracts (far edge narrower)
ZONE_RIGHT_PERIPHERAL (2)	-0.15	Left edge contracts (far edge narrower)

The WARP_STRENGTH = 0.15f constant controls how aggressively the far edge of each peripheral zone is compressed. A value of 0 produces no warp; 1.0 produces an extreme trapezoid that would look distorted. The 0.15 setting provides a subtle but perceptible perspective correction that matches the curved optical FOV of the INMO AIR3 glasses (~31° horizontal).

Sources: portal/wayfire-plugins/portal-spatial-warp/plugin.cpp#L37-L83, portal/wayfire-plugins/portal-spatial-config.h#L36-L41

View Lifecycle and Transform Application

The plugin operates entirely through Wayfire’s signal system. On view_mapped_signal, it queries the zone ID via the FFI function, then attaches (or reuses) a view_3d_transformer_t named "perspective-warp" to the view’s transform node. The transformer’s view_proj matrix is set to the trapezoidal warp (or identity for center zone), while translation, rotation, and scaling are left at identity — positioning is handled by the separate portal-spatial plugin. On view_unmapped_signal, the view’s entry is erased from the transformed_views tracking map.

Sources: portal/wayfire-plugins/portal-spatial-warp/plugin.cpp#L118-L165

SHA-256 Sidecar Integrity Verification

Before dlopen-ing libportal_spatial_plugin.so, the plugin verifies its integrity against a SHA-256 sidecar file (<path>.sha256). If the sidecar exists and sha256sum -c fails, the plugin refuses to load and logs an error. If no sidecar is present, verification is skipped (treated as trusted). This mechanism prevents loading a tampered or corrupted shared library into the compositor process.

Sources: portal/wayfire-plugins/portal-spatial-warp/plugin.cpp#L21-L35, portal/wayfire-plugins/portal-spatial-warp/plugin.cpp#L86-L97

---

Zone Model and Canvas Geometry

The warp plugin operates on a 2560×1080 virtual canvas (the GLASSES-1 output) divided into three equal zones. The geometry is defined in portal-spatial-config.h as compile-time constants and mirrored in the runtime configuration file spatial-warp.toml.

flowchart LR
    subgraph Canvas["GLASSES-1: 2560×1080"]
        LP["Left Peripheral\nx ∈ [0, 853)\nWarp: +0.15"]
        CT["Center\nx ∈ [853, 1706)\nNo Warp"]
        RP["Right Peripheral\nx ∈ [1706, 2560]\nWarp: -0.15"]
    end

The 2560 / 3 division yields 853.33, integer-truncated to 853px per zone. The right peripheral zone absorbs the 1px rounding remainder, making it 854px wide — visually imperceptible but documented to prevent confusion. The canvas dimensions match the INMO AIR3 optical FOV and reduce H.265 encoder bandwidth by approximately 33% compared to the previous 3840×1080 layout.

Sources: portal/wayfire-plugins/portal-spatial-config.h#L14-L41, portal/compositor/spatial-warp.toml#L1-L32

---

FFI Safety Architecture

Both the warp crate and the spatial crate implement rigorous panic-safety boundaries at the C ABI surface, recognizing that a Rust panic unwinding into C code is undefined behavior.

Panic Boundary Guard (portal-warp)

Every #[no_mangle] extern "C" function in the warp crate is wrapped in ffi_boundary_guard, which uses catch_unwind with AssertUnwindSafe to intercept panics. On panic, it logs via tracing::error! with the payload message and returns a safe fallback value (null pointer, zero, or unit) to the C caller. This ensures that even if the internal Rust logic panics (e.g., due to a numerical edge case not caught by the solver’s degeneracy checks), the compositor process will not crash.

Sources: portal/warp/src/lib.rs#L60-L80

Tracked Pointer Validation (portal-spatial)

The spatial crate uses a Tracked<T> wrapper that stores each heap allocation with a magic number and an atomic “live” flag. When C code passes a pointer back through FFI, Tracked::as_ref atomically checks null + magic + liveness before returning a reference. This prevents use-after-free, double-free, and type confusion attacks where a C caller might pass a stale or mismatched pointer.

Sources: portal/spatial/src/ffi/homography.rs#L17-L37, portal/spatial/src/ffi/types.rs#L34-L50

---

Shader Compilation Pipeline

The shader module provides a trait-based abstraction (GlFunctions) that decouples GLSL compilation from any specific GL function loader. The C++ shim is expected to provide the concrete implementation by populating function pointers at runtime. The compilation pipeline follows the standard GL pattern with explicit error checking at every stage:

1. Pre-existing error check — glGetError() is called before compilation; any non-zero result aborts immediately to avoid attributing a prior error to this shader
2. Shader creation — glCreateShader returning 0 indicates GL context or resource exhaustion
3. Compile status — after glCompileShader, the info log is captured before the shader object is deleted on failure
4. Link status — vertex and fragment shaders are deleted after linking (still referenced by the program object)
5. Post-link error sweep — a final glGetError() catches any driver-level issue

The ShaderProgram wrapper struct provides idempotent compilation (compiles only once) and an explicit delete method. Notably, the WarpRenderer struct deliberately does not implement Drop — GL resource cleanup is the caller’s responsibility via the cleanup() method. A custom Semgrep rule (warp-renderer-no-drop) was introduced to flag this pattern for any struct that owns GL handles, preventing silent GPU memory leaks.

Sources: portal/warp/src/shader.rs#L195-L343, .semgrep-rules/warp-renderer-no-drop/warp-renderer-no-drop.yaml#L1-L19

---

Testing Strategy

The warp crate’s test suite covers three verification dimensions. Unit tests in homography/tests.rs validate identity transforms, translations, scaling, 90° rotation, known perspective mappings, and both collinear source/destination rejection paths. Property-based tests in proptests.rs use proptest to verify that identity mappings preserve corners across random scales (1.0–1000.0) and that arbitrary collinear point configurations are always rejected with the correct error variant. Renderer tests use a MockGl struct that implements both GlFunctions and GlRenderFunctions traits without a real GL context, recording all calls and providing configurable failure injection for each pipeline stage (shader creation failure, compile failure, link failure, attribute location failure, resource allocation failure).

Test Layer	Module	Coverage
Unit (homography)	homography/tests.rs	Identity, translation, scale, rotation, perspective, collinear rejection
Property	proptests.rs	Random-scale identity preservation, random collinear rejection
Unit (shader)	shader/tests.rs	Source validation, version directives, uniform presence, compile/link failure paths
Unit (renderer)	renderer/tests.rs	Full pipeline with MockGl, resource allocation failure, attribute/uniform location failure
Unit (FFI)	lib.rs (inline)	WarpState size/alignment, null pointer handling, identity initialization

Sources: portal/warp/src/homography/tests.rs#L1-L161, portal/warp/src/proptests.rs#L1-L43, portal/warp/src/renderer/tests.rs#L1-L58, portal/warp/src/shader/tests.rs#L1-L200

---

Dependency Profile

The warp crate maintains a minimal dependency surface — only nalgebra for linear algebra, thiserror for error derivation, and tracing for structured logging. The crate-type is cdylib, producing a shared library (libportal_warp.so) suitable for FFI consumption.

Dependency	Version	Purpose
nalgebra	0.33	DMatrix/DVector for DLT, Matrix3 for homography, SymmetricEigen for eigendecomposition
thiserror	workspace	#[derive(Error)] for WarpError
tracing	workspace	Structured panic logging at FFI boundary
proptest (dev)	workspace	Property-based testing for solver invariants

Sources: portal/warp/Cargo.toml#L1-L22

---

Relationship to Head Tracking and ATW

The warp layer documented here provides static spatial correction — compensating for the fixed geometric relationship between the compositor canvas and the AR glasses optics. This is distinct from Asynchronous Timewarp (ATW), which dynamically counteracts head movement latency by re-projecting the rendered frame based on the latest IMU pose. The INMO AIR3’s own SDK implements ATW through quaternion-based view ray transformation (calcAtwMat, atwModelViewMat), while Portal’s current architecture performs spatial warp at the compositor stage and delegates timewarp to the glasses-side runtime. The homography solver infrastructure in the warp crate is designed to support both modes — the DLT solver can compute corrections from runtime head pose data, not just static zone geometry — but the production code path currently uses only the static zone-based trapezoidal warp.

Sources: portal/wayfire-plugins/portal-spatial-warp/plugin.cpp#L60-L83, docs/world-locking-analysis.md#L17-L58

---

Next Steps

• To understand how views are assigned to zones in the first place, see Spatial Domain Model: Zones, Dimensions, and Assignment Policies.
• For the Rust-to-C++ FFI pattern used by the spatial plugins, see Wayfire Plugin Integration: Rust FFI via C++ Shims.
• For the GPU rendering context (EGL paths, Mesa Turnip driver), see GPU Rendering Strategy: wgpu, EGL Paths, and Mesa Turnip Driver.
• For how the warped compositor output is encoded and streamed to the glasses, see End-to-End Streaming: DMA-BUF to H.265 Hardware Encode to RTP/UDP.