Choosing RMSE Thresholds by Survey Class

A project manager forwards you a check-shot report: the drone survey’s horizontal RMSE is 0.045 m and vertical RMSE is 0.08 m. Is that a pass? The honest answer is it depends on what the deliverable is — 0.08 m vertical is comfortable for a topographic contour map, marginal for a stockpile volume, and an outright failure for an as-built foundation check. The single most common accuracy-governance mistake in UAV mapping is applying one hard-coded RMSE number to every job regardless of what the client actually contracted. This page shows how to derive horizontal and vertical thresholds per survey class, encode them in a table, and wire that table into an automated acceptance gate so a run that misses its class tolerance fails loudly before anyone signs a deliverable.

Why one RMSE threshold cannot serve every deliverable

RMSE is the root-mean-square of the residuals between checked coordinates and their surveyed truth. For a single axis of nn check points it is

RMSE=1n∑i=1n(x^i−xi)2\text{RMSE} = \sqrt{\frac{1}{n}\sum_{i=1}^{n}\left(\hat{x}_i - x_i\right)^2}

where x^i\hat{x}_i is the computed coordinate and xix_i the surveyed truth. Horizontal accuracy combines the easting and northing components in quadrature,

RMSEr=RMSEx2+RMSEy2\text{RMSE}_r = \sqrt{\text{RMSE}_x^{2} + \text{RMSE}_y^{2}}

and vertical accuracy is reported separately because height error is driven by different physics — weak vertical geometry in nadir imagery and geoid/datum handling rather than tie-point matching. The reason a single threshold fails is that these two numbers must be judged against the tolerance the deliverable can absorb, and that tolerance varies by an order of magnitude across classes. A contour map with a 0.25 m contour interval tolerates several centimetres of vertical error invisibly; a volumetric stockpile survey converts every centimetre of vertical bias directly into tonnes of miscounted material; an as-built engineering check must resolve position tighter than the construction tolerance it verifies. Deriving the number is the point of the parent setting accuracy thresholds for survey projects workflow; this page turns those derivations into a class-keyed gate.

Survey-class thresholds feeding a single RMSE acceptance gate Three deliverable-class boxes on the left — topographic mapping with horizontal tolerance 0.05 metres and vertical 0.10 metres, volumetric or stockpile with 0.03 and 0.05 metres, and as-built or engineering with 0.02 and 0.03 metres — each feed into a central decision gate that asks whether the measured RMSE is less than or equal to the selected class threshold. If it passes, the flow reaches a green accept-deliverable box; if it fails, it reaches a gold reject or re-fly box. Topographic map H ≤ 0.05 m · V ≤ 0.10 m Volumetric / stockpile H ≤ 0.03 m · V ≤ 0.05 m As-built / engineering H ≤ 0.02 m · V ≤ 0.03 m RMSE ≤ class threshold? Accept deliverable signed off Reject / re-fly densify GCP · re-process pass fail Accuracy classes against what each deliverable is used for A ladder of survey classes plotted against required horizontal accuracy, from a coarse planning-grade map at half a metre down to as-built engineering work at two centimetres. Beside each rung is the decision the deliverable supports and therefore the consequence of being wrong: a planning map misplaces a proposed boundary on paper, a volumetric survey misprices a stockpile, and an as-built survey misplaces a structure that has already been built. A note observes that the cost of an error scales far faster than the cost of achieving the accuracy, which is why the class is chosen from the use, never from the equipment. planning / reconnaissance horizontal ≤ 0.50 m topographic mapping horizontal ≤ 0.05 m volumetric / stockpile horizontal ≤ 0.03 m as-built / engineering horizontal ≤ 0.02 m supports: where might this go? an error moves a line on a drawing supports: what is where, on the ground? an error misplaces a mapped feature supports: how much material is there? an error misprices the stockpile supports: was it built where specified? an error disputes a structure already poured Pick the class from the decision the deliverable supports, never from the equipment available.

Figure 2 — The threshold is a property of the decision, not of the drone. Each rung down costs more control and more flying, and each rung down also changes what an error costs whoever relies on the result.

Minimal reproducible solution

The solution has two parts: a threshold table that encodes the class-specific tolerances, and a gate function that looks up the right row and evaluates a measured horizontal and vertical RMSE against it. The thresholds below are engineering starting points for RTK/PPK-flown surveys with checked control — tighten them to your contract and local specification, never loosen them to make a marginal run pass.

Survey class Horizontal RMSE ≤ Vertical RMSE ≤ Typical driver
Reconnaissance / orthophoto 0.10 m 0.20 m visual context, no measurement
Topographic mapping 0.05 m 0.10 m contour interval, planimetric detail
Volumetric / stockpile 0.03 m 0.05 m height error → volume error
As-built / engineering 0.02 m 0.03 m construction tolerance verification

The gate keys on the survey class with a match statement, computes the combined horizontal RMSE from the per-axis values in quadrature, and returns a structured pass/fail with the reason — so a failing run cannot be silently rounded into acceptance.

from dataclasses import dataclass
from math import hypot

# Class -> (horizontal RMSE limit, vertical RMSE limit) in metres.
THRESHOLDS: dict[str, tuple[float, float]] = {
    "recon":        (0.10, 0.20),
    "topographic":  (0.05, 0.10),
    "volumetric":   (0.03, 0.05),
    "as_built":     (0.02, 0.03),
}

@dataclass(frozen=True)
class AcceptanceResult:
    survey_class: str
    passed: bool
    rmse_h: float
    rmse_v: float
    reason: str

def acceptance_gate(survey_class: str, rmse_x: float, rmse_y: float,
                    rmse_z: float) -> AcceptanceResult:
    """Evaluate measured per-axis RMSE against the class thresholds."""
    match survey_class:
        case cls if cls in THRESHOLDS:
            h_limit, v_limit = THRESHOLDS[cls]
        case _:
            raise KeyError(f"unknown survey_class {survey_class!r}; "
                           f"valid: {sorted(THRESHOLDS)}")

    rmse_h = hypot(rmse_x, rmse_y)          # combine E/N in quadrature
    h_ok = rmse_h <= h_limit
    v_ok = rmse_z <= v_limit
    if h_ok and v_ok:
        reason = "within class tolerance"
    elif not h_ok and not v_ok:
        reason = f"horizontal {rmse_h:.3f}>{h_limit} and vertical {rmse_z:.3f}>{v_limit}"
    elif not h_ok:
        reason = f"horizontal {rmse_h:.3f} m exceeds {h_limit} m"
    else:
        reason = f"vertical {rmse_z:.3f} m exceeds {v_limit} m"
    return AcceptanceResult(survey_class, h_ok and v_ok, rmse_h, rmse_z, reason)

The design choice that matters is returning a structured result rather than a bare boolean: the reason string tells the operator which axis failed, which immediately narrows the fix — a horizontal failure points at control distribution or a CRS problem, a vertical failure points at datum handling or weak height geometry.

Edge-case matrix

These are the inputs that make a naive gate give the wrong verdict.

Input variant Downstream symptom if unchecked Expected handling
Horizontal passes, vertical fails orthomosaic accepted, DSM/volumes wrong evaluate axes independently, never a single blended RMSE
rmse_x and rmse_y reported separately operator averages instead of combining combine in quadrature with hypot, not arithmetic mean
Unknown survey_class string gate defaults to a lax class, bad data passes raise KeyError, never fall back to a permissive default
RMSE computed on GCPs used in adjustment over-optimistic — measures fit, not accuracy compute on independent check points, not control points
Thresholds hard-coded per script drift between jobs, no audit trail keep one central table, load it, log the version used
Vertical RMSE in feet, limit in metres unit mismatch silently passes/fails normalise units at ingestion; assert metres before the gate

The check-point versus control-point distinction is the subtle one: if you compute RMSE against the same markers the bundle adjustment already fitted, you are measuring how well the solver minimised its own cost function, not independent accuracy. Reserve a subset of surveyed markers as check points, exclude them from adjustment, and run the gate on those.

The error budget behind a single threshold A stacked budget showing the independent error sources that combine into a delivered accuracy figure. The control survey itself contributes a base uncertainty. Target identification in the imagery contributes a further term that depends on ground sample distance. The bundle adjustment contributes its own residual. The rasterisation to a finite grid contributes half a cell. These combine in quadrature, not by addition, so the largest term dominates and shaving the smallest ones is wasted effort. A worked total is shown reaching just under the class threshold. control survey target identification bundle adjustment raster cell (half a cell) combined in quadrature 0.010 m — RTK base and rover 0.018 m — ≈ 1 px at 1.8 cm GSD 0.013 m — block residual 0.009 m — 1.8 cm cell 0.026 m against a 0.030 m class limit Terms combine as the square root of the sum of squares, so the largest one dominates. Halving the smallest term here changes the total by under a millimetre; flying lower to improve GSD changes it by several.

Figure 3 — A threshold is met or missed by a budget, not by a single stage. Quadrature is why effort belongs on the largest term, and why “we could tighten the control survey” is usually the wrong answer.

Verification snippet

Assert that the gate accepts a clean run at its class and rejects a run that busts vertical tolerance, so the gate’s own logic is proven before it guards a real deliverable.

# A topographic run: horizontal 0.032 m combined, vertical 0.07 m — should pass.
ok = acceptance_gate("topographic", rmse_x=0.021, rmse_y=0.024, rmse_z=0.07)
assert ok.passed, ok.reason
assert ok.rmse_h <= 0.05 and ok.rmse_v <= 0.10

# The same numbers judged as volumetric must fail on vertical (limit 0.05 m).
strict = acceptance_gate("volumetric", rmse_x=0.021, rmse_y=0.024, rmse_z=0.07)
assert not strict.passed
assert "vertical" in strict.reason
print("gate verified:", ok.reason, "|", strict.reason)

The second assertion is the whole point of class-keyed thresholds: identical measured accuracy is a pass for one deliverable and a fail for another, and the gate encodes that difference instead of leaving it to a human eyeballing a report.

Note also that horizontal and vertical thresholds are rarely equal, and that the vertical one is usually the binding constraint. Photogrammetric height is derived from the intersection of rays that are nearly parallel on a nadir flight, so vertical accuracy is characteristically one and a half to two times worse than horizontal from the same block. A specification that quotes a single figure for both is either quoting the horizontal one and expecting the vertical to follow, or it has not been read carefully — and it is worth resolving which before the flight rather than after the check points are computed.

When to escalate

The gate decides pass/fail; when it fails, the fix lives in the sibling clusters.

← Setting Accuracy Thresholds for Survey Projects