Reconciling Volume Differences Between Flights

The monthly report says the stockpile lost 640 m³. The site manager says nothing left the yard. Somebody is wrong, and the argument that follows usually runs on assertion rather than evidence, because the two volumes are single numbers with no visible inputs.

There are exactly five things that can make two volumes of an unchanged pile differ, and four of them are processing artefacts. Attributing a difference is a mechanical procedure once the run records described in computing volumes and stockpiles in Python exist, and it takes about two minutes.

The five causes, in the order to check them

The boundary changed. Cheapest to check and the most common cause. If the two runs used different polygons, the comparison is not like-for-like and nothing else needs investigating until it is redone.

The base surface changed. A base re-fitted per survey moves as the pile moves. A prior terrain model that was replaced, or a fixed level that was edited, does the same thing more visibly.

The vertical datum shifted. A change in the geoid model, the PROJ grid availability, or the ground control used will offset one entire survey relative to the other. The signature is unmistakable: the hardstanding around the pile differs by the same amount as the pile.

The reconstruction quality changed. More NoData inside the boundary means less volume counted. Dark wet material, low sun, or a thinner flight plan all produce this, and it always reduces the apparent volume.

Material actually moved. What is left after the other four are excluded.

Decision procedure for attributing a volume difference A top-down decision tree. Starting from an unexplained volume difference, the first test compares the boundary hashes of the two runs; a mismatch attributes the difference to the boundary. If they match, the base parameters are compared; a mismatch attributes it to the base. If those match, the elevation difference over stable hardstanding is measured; a non-zero median attributes it to the vertical datum. If that is near zero, the void fractions are compared; a large increase attributes it to reconstruction quality. Only if all four tests pass is the difference attributed to material movement, and the tree notes that this is where the site conversation should start. 1 · boundary hashes equal? 2 · base parameters equal? 3 · hardstanding agrees? 4 · void fractions similar? 5 · difference > 2σ? material moved now talk to the site boundary changed — redo base moved — freeze it datum offset — co-register reconstruction worse — re-fly

Figure 1 — Four tests, in cost order. The site conversation is the last step, not the first.

Minimal reproducible solution

Given two run records, the attribution is a comparison over their stored inputs.

import numpy as np


def attribute_difference(run_a: dict, run_b: dict,
                         hardstanding_diff: np.ndarray,
                         sigma_total_m3: float) -> dict:
    """Explain a volume difference between two runs, in test order.

    run_a / run_b are the volume records: volume, inputs (boundary digest,
    base mode and parameters, cell size), and void fraction.
    """
    dv = run_b["volume_m3"] - run_a["volume_m3"]
    findings = []

    if run_a["inputs"]["boundary"]["sha256_16"] != run_b["inputs"]["boundary"]["sha256_16"]:
        findings.append("boundary geometry differs — not a like-for-like comparison")

    if (run_a["inputs"]["base_mode"] != run_b["inputs"]["base_mode"]
            or run_a["inputs"]["base_params"] != run_b["inputs"]["base_params"]):
        findings.append("base surface differs between runs")

    stable = hardstanding_diff[np.isfinite(hardstanding_diff)]
    datum_shift = float(np.median(stable)) if stable.size else float("nan")
    if abs(datum_shift) > 0.02:
        findings.append(f"surfaces differ by {datum_shift:+.3f} m on stable ground — "
                        "a datum or alignment offset, not material")

    void_delta = run_b["void_fraction"] - run_a["void_fraction"]
    if void_delta > 0.03:
        findings.append(f"void fraction rose {void_delta:+.1%} — volume is under-counted")

    significant = abs(dv) > 2 * sigma_total_m3
    return {
        "delta_m3": dv,
        "significant": bool(significant and not findings),
        "artefacts": findings,
        "verdict": ("material movement" if significant and not findings
                    else "within noise" if not significant and not findings
                    else "processing artefact — resolve before reporting"),
    }

The ordering is not cosmetic. Each test is cheaper than the one after it, and each one, if it fires, invalidates everything below — there is no point measuring a datum shift between two runs that used different boundaries.

Reconciling two volumes: what to check, in order Four checks are listed in the order that resolves a disagreement fastest. First, compare the boundary polygons, because a differently drawn toe accounts for more disagreements than everything else combined. Second, compare the base surface method, since an interpolated pad and a measured one differ systematically. Third, compare the grid cell size, which changes how the toe slope is integrated. Fourth, compare the vertical datum, which is rarer and larger when it happens. A note states that only after all four match is a remaining difference actually material change. 1. the boundary polygons a differently drawn toe explains more disagreements than the rest combined 2. the base surface method an interpolated pad and a measured one differ systematically 3. the grid cell size changes how the toe slope is integrated 4. the vertical datum rarer, and much larger when it happens Only once all four match is the remaining difference actually material.

Figure 3 — Check in this order and most disagreements resolve at step one.

Edge-case matrix

Observation Likely cause Confirming check
Hardstanding differs by a constant Vertical datum or control Median difference on stable ground
Hardstanding agrees, pile differs uniformly Base surface moved Compare stored base parameters
Difference is a clean step month to month Boundary re-traced Compare boundary digests
Pile “shrinks” in winter More NoData from wet, dark material Compare void fractions
Paired cut and fill on opposite slopes Horizontal misalignment Difference map shows a dipole pattern
Gradual drift over many months Base re-fitted each time as the pile grew Plot the fitted plane coefficients over time
Difference smaller than 2σ Nothing detectable happened Report as “no significant change”
Only one lobe of a pile changed Genuine movement Difference map is localised

The dipole pattern in the fifth row is worth learning to recognise on sight. A horizontal shift between two surveys produces material apparently removed from one side of every slope and added to the other, summing to nearly zero overall while making the difference map look dramatic. It is an alignment fault, and the fix is in aligning two epochs with ICP before differencing.

Verification snippet

Once the attribution says “material moved”, the claim should be checked against the shape of the change rather than only its total.

import numpy as np


def movement_plausibility(diff: np.ndarray, mask: np.ndarray,
                          cell: float, limit: float) -> dict:
    """Is the change spatially coherent, or scattered noise that happened to sum?"""
    signif = mask & np.isfinite(diff) & (np.abs(diff) > limit)
    if not signif.any():
        return {"verdict": "no cells exceed the detection limit"}

    from scipy import ndimage
    labelled, n = ndimage.label(signif)
    sizes = ndimage.sum(signif, labelled, range(1, n + 1)) * cell ** 2
    largest = float(sizes.max()) if n else 0.0
    coherent = largest / (signif.sum() * cell ** 2)

    return {
        "changed_area_m2": float(signif.sum() * cell ** 2),
        "clusters": int(n),
        "largest_cluster_m2": largest,
        "coherence": float(coherent),
        "verdict": ("coherent movement in one area" if coherent > 0.5
                    else "scattered — check for noise or misalignment"),
    }

Real material movement is spatially coherent: a loader works one face, and the change appears as one connected region. A difference spread as thousands of small clusters across the whole pile is almost always a processing artefact that survived the earlier tests, most often a resolution or interpolation change.

Coherent movement against scattered difference, on the same pile outline Two plan views of the same stockpile outline. On the left a single connected region on one face is shaded, representing material worked by a loader, with ninety-one percent of the changed area in one cluster. On the right the same total changed area is broken into dozens of small scattered patches across the whole pile, with the largest cluster holding only six percent. A note states that both sum to the same volume difference and only the left one is plausible as material movement. coherent — 91 % in one cluster scattered — largest is 6 % unchanged Both sum to −640 m³. Only one is a loader. Cluster coherence is the cheapest test that distinguishes them, and it needs no extra data.

Figure 2 — The shape of the change is evidence. A total on its own is not.

Writing the reconciliation into the monthly report

The point of the procedure is to stop these conversations recurring, which means the output belongs in the report rather than in a technician’s head. A useful monthly entry is three lines: the difference, the verdict, and the evidence that supports it.

A difference declared “within noise” should state the detection threshold, because a client reading “no significant change” reasonably wants to know what would have been significant. A difference attributed to an artefact should never be reported as a volume change at all — it should be corrected and the run repeated, with a note that the earlier figure was superseded.

The cumulative series deserves one further discipline. Plot the running total alongside the deliveries and removals the site records independently, and reconcile at the year end rather than every month. Individual months are noisy; a twelve-month series against independent tallies is the strongest evidence a survey programme can produce that its numbers are sound, and it costs nothing beyond keeping the records.

def report_lines(attr: dict, detection_m3: float) -> list[str]:
    """Three lines for the monthly report, from the attribution result."""
    lines = [f"Change since previous survey: {attr['delta_m3']:+,.0f} m³"]
    if attr["artefacts"]:
        lines.append("SUPERSEDED — processing differences found: "
                     + "; ".join(attr["artefacts"]))
        lines.append("This figure is not comparable and has been recomputed.")
    elif attr["significant"]:
        lines.append(f"Attributed to material movement (detection limit "
                     f"±{detection_m3:,.0f} m³).")
        lines.append("Boundary, base surface and datum verified unchanged.")
    else:
        lines.append(f"No significant change: below the ±{detection_m3:,.0f} m³ "
                     "detection limit for this pile.")
        lines.append("Inputs verified identical to the previous survey.")
    return lines

When to escalate

  • Everything checks out and the site still disagrees. Compare what each side measures. A truck tally counts loose material; a surface integral measures in-place volume, and the bulking factor between them is commonly 15–30 %. Neither figure is wrong.
  • The datum shifted and the earlier surveys cannot be recovered. The series has a break in it. Document the break rather than splicing across it, and re-baseline from the corrected survey onward.
  • The difference is significant, coherent and unexplained by the site. That is a security or process finding, not a survey one. Hand it over with the difference map rather than trying to settle it in processing.

← Computing Volumes and Stockpiles in Python