Choosing RMSE Thresholds by Survey Class

A project manager forwards you a check-shot report: the drone survey’s horizontal RMSE is 0.045 m and vertical RMSE is 0.08 m. Is that a pass? The honest answer is it depends on what the deliverable is — 0.08 m vertical is comfortable for a topographic contour map, marginal for a stockpile volume, and an outright failure for an as-built foundation check. The single most common accuracy-governance mistake in UAV mapping is applying one hard-coded RMSE number to every job regardless of what the client actually contracted. This page shows how to derive horizontal and vertical thresholds per survey class, encode them in a table, and wire that table into an automated acceptance gate so a run that misses its class tolerance fails loudly before anyone signs a deliverable.

Why one RMSE threshold cannot serve every deliverable

RMSE is the root-mean-square of the residuals between checked coordinates and their surveyed truth. For a single axis of nn check points it is

RMSE=1n∑i=1n(x^i−xi)2\text{RMSE} = \sqrt{\frac{1}{n}\sum_{i=1}^{n}\left(\hat{x}_i - x_i\right)^2}

where x^i\hat{x}_i is the computed coordinate and xix_i the surveyed truth. Horizontal accuracy combines the easting and northing components in quadrature,

RMSEr=RMSEx2+RMSEy2\text{RMSE}_r = \sqrt{\text{RMSE}_x^{2} + \text{RMSE}_y^{2}}

and vertical accuracy is reported separately because height error is driven by different physics — weak vertical geometry in nadir imagery and geoid/datum handling rather than tie-point matching. The reason a single threshold fails is that these two numbers must be judged against the tolerance the deliverable can absorb, and that tolerance varies by an order of magnitude across classes. A contour map with a 0.25 m contour interval tolerates several centimetres of vertical error invisibly; a volumetric stockpile survey converts every centimetre of vertical bias directly into tonnes of miscounted material; an as-built engineering check must resolve position tighter than the construction tolerance it verifies. Deriving the number is the point of the parent setting accuracy thresholds for survey projects workflow; this page turns those derivations into a class-keyed gate.

Survey-class thresholds feeding a single RMSE acceptance gate Three deliverable-class boxes on the left — topographic mapping with horizontal tolerance 0.05 metres and vertical 0.10 metres, volumetric or stockpile with 0.03 and 0.05 metres, and as-built or engineering with 0.02 and 0.03 metres — each feed into a central decision gate that asks whether the measured RMSE is less than or equal to the selected class threshold. If it passes, the flow reaches a green accept-deliverable box; if it fails, it reaches a gold reject or re-fly box. Topographic map H ≤ 0.05 m · V ≤ 0.10 m Volumetric / stockpile H ≤ 0.03 m · V ≤ 0.05 m As-built / engineering H ≤ 0.02 m · V ≤ 0.03 m RMSE ≤ class threshold? Accept deliverable signed off Reject / re-fly densify GCP · re-process pass fail

Minimal reproducible solution

The solution has two parts: a threshold table that encodes the class-specific tolerances, and a gate function that looks up the right row and evaluates a measured horizontal and vertical RMSE against it. The thresholds below are engineering starting points for RTK/PPK-flown surveys with checked control — tighten them to your contract and local specification, never loosen them to make a marginal run pass.

Survey class Horizontal RMSE ≤ Vertical RMSE ≤ Typical driver
Reconnaissance / orthophoto 0.10 m 0.20 m visual context, no measurement
Topographic mapping 0.05 m 0.10 m contour interval, planimetric detail
Volumetric / stockpile 0.03 m 0.05 m height error → volume error
As-built / engineering 0.02 m 0.03 m construction tolerance verification

The gate keys on the survey class with a match statement, computes the combined horizontal RMSE from the per-axis values in quadrature, and returns a structured pass/fail with the reason — so a failing run cannot be silently rounded into acceptance.

from dataclasses import dataclass
from math import hypot

# Class -> (horizontal RMSE limit, vertical RMSE limit) in metres.
THRESHOLDS: dict[str, tuple[float, float]] = {
    "recon":        (0.10, 0.20),
    "topographic":  (0.05, 0.10),
    "volumetric":   (0.03, 0.05),
    "as_built":     (0.02, 0.03),
}

@dataclass(frozen=True)
class AcceptanceResult:
    survey_class: str
    passed: bool
    rmse_h: float
    rmse_v: float
    reason: str

def acceptance_gate(survey_class: str, rmse_x: float, rmse_y: float,
                    rmse_z: float) -> AcceptanceResult:
    """Evaluate measured per-axis RMSE against the class thresholds."""
    match survey_class:
        case cls if cls in THRESHOLDS:
            h_limit, v_limit = THRESHOLDS[cls]
        case _:
            raise KeyError(f"unknown survey_class {survey_class!r}; "
                           f"valid: {sorted(THRESHOLDS)}")

    rmse_h = hypot(rmse_x, rmse_y)          # combine E/N in quadrature
    h_ok = rmse_h <= h_limit
    v_ok = rmse_z <= v_limit
    if h_ok and v_ok:
        reason = "within class tolerance"
    elif not h_ok and not v_ok:
        reason = f"horizontal {rmse_h:.3f}>{h_limit} and vertical {rmse_z:.3f}>{v_limit}"
    elif not h_ok:
        reason = f"horizontal {rmse_h:.3f} m exceeds {h_limit} m"
    else:
        reason = f"vertical {rmse_z:.3f} m exceeds {v_limit} m"
    return AcceptanceResult(survey_class, h_ok and v_ok, rmse_h, rmse_z, reason)

The design choice that matters is returning a structured result rather than a bare boolean: the reason string tells the operator which axis failed, which immediately narrows the fix — a horizontal failure points at control distribution or a CRS problem, a vertical failure points at datum handling or weak height geometry.

Edge-case matrix

These are the inputs that make a naive gate give the wrong verdict.

Input variant Downstream symptom if unchecked Expected handling
Horizontal passes, vertical fails orthomosaic accepted, DSM/volumes wrong evaluate axes independently, never a single blended RMSE
rmse_x and rmse_y reported separately operator averages instead of combining combine in quadrature with hypot, not arithmetic mean
Unknown survey_class string gate defaults to a lax class, bad data passes raise KeyError, never fall back to a permissive default
RMSE computed on GCPs used in adjustment over-optimistic — measures fit, not accuracy compute on independent check points, not control points
Thresholds hard-coded per script drift between jobs, no audit trail keep one central table, load it, log the version used
Vertical RMSE in feet, limit in metres unit mismatch silently passes/fails normalise units at ingestion; assert metres before the gate

The check-point versus control-point distinction is the subtle one: if you compute RMSE against the same markers the bundle adjustment already fitted, you are measuring how well the solver minimised its own cost function, not independent accuracy. Reserve a subset of surveyed markers as check points, exclude them from adjustment, and run the gate on those.

Verification snippet

Assert that the gate accepts a clean run at its class and rejects a run that busts vertical tolerance, so the gate’s own logic is proven before it guards a real deliverable.

# A topographic run: horizontal 0.032 m combined, vertical 0.07 m — should pass.
ok = acceptance_gate("topographic", rmse_x=0.021, rmse_y=0.024, rmse_z=0.07)
assert ok.passed, ok.reason
assert ok.rmse_h <= 0.05 and ok.rmse_v <= 0.10

# The same numbers judged as volumetric must fail on vertical (limit 0.05 m).
strict = acceptance_gate("volumetric", rmse_x=0.021, rmse_y=0.024, rmse_z=0.07)
assert not strict.passed
assert "vertical" in strict.reason
print("gate verified:", ok.reason, "|", strict.reason)

The second assertion is the whole point of class-keyed thresholds: identical measured accuracy is a pass for one deliverable and a fail for another, and the gate encodes that difference instead of leaving it to a human eyeballing a report.

When to escalate

The gate decides pass/fail; when it fails, the fix lives in the sibling clusters.

← Setting Accuracy Thresholds for Survey Projects