Deciding When Radiometric Calibration Is Worth It

A new multispectral programme has to decide whether to calibrate before it flies, because the decision is not reversible after the fact — the panel images either exist or they do not. It is usually made by whoever has the strongest opinion, and the opinions divide along lines that have little to do with the work being done.

There is a better procedure: four questions with concrete answers, and one measurement that settles the remaining doubt. This page sets it out, as the operational companion to calibrated vs uncalibrated multispectral workflows.

# The four questions

Will any two flights be compared? If the deliverable is ever a change, a trend or a time series, calibration is required. If every deliverable is a single map read on its own, it is not.

Will values be compared against anything external? A published threshold, another operator’s data, a laboratory measurement. If so, calibration is necessary — though, as the parent page notes, rarely sufficient, because band centres differ between sensors.

Will the flights be under stable light? A programme flying only in settled conditions within an hour of solar noon has already removed much of what calibration corrects. One flying whenever the client asks, under whatever sky, has not.

Will the panel be captured reliably? A partially calibrated archive is harder to use than either a complete one or none, so this is a question about process rather than intent.

Figure 1 — Four questions, one of which most programmes answer “yes” to immediately.

# Minimal reproducible solution

Where the answers leave genuine doubt — typically a programme doing within-flight ranking under mostly stable light — the doubt is measurable on a single pilot flight processed both ways.

import numpy as np
from scipy.stats import spearmanr


def pilot_decision(uncal_by_plot: dict[str, float],
                   cal_by_plot: dict[str, float],
                   *, rank_threshold: float = 0.95,
                   top_agreement_threshold: float = 0.85) -> dict:
    """Decide from one pilot flight whether calibration changes any decision.

    The test is deliberately about decisions rather than values. Calibration
    changes every number by construction; what matters is whether it changes
    which plots a reader would act on.
    """
    plots = sorted(set(uncal_by_plot) & set(cal_by_plot))
    if len(plots) < 12:
        return {"verdict": "inconclusive", "note": "need at least 12 common plots"}

    u = np.array([uncal_by_plot[p] for p in plots])
    c = np.array([cal_by_plot[p] for p in plots])
    rho = float(spearmanr(u, c).statistic)

    k = max(len(plots) // 5, 1)
    top_u = set(np.array(plots)[np.argsort(-u)][:k])
    top_c = set(np.array(plots)[np.argsort(-c)][:k])
    agreement = len(top_u & top_c) / k

    changes = rho < rank_threshold or agreement < top_agreement_threshold
    return {
        "plots": len(plots), "rank_correlation": rho,
        "top_quintile_agreement": float(agreement),
        "verdict": "calibrate" if changes else "uncalibrated is sufficient here",
        "note": ("calibration reorders the plots a reader would act on"
                 if changes else
                 "calibration changes values but not decisions on this flight"),
    }

Running this on one flight is an afternoon’s work and replaces an argument that otherwise recurs at every planning meeting. It is worth repeating once per season, because the answer depends on the crop stage as much as on the method.

Figure 3 — Decide before the flight; the data cannot be calibrated afterwards.

# Edge-case matrix

The contractor row is the one most often got wrong. Data flown for a client becomes their archive, and a contractor has no visibility of what it will be compared against in two years. Calibrating by default is the only defensible position when the future use is unknown.

# What the decision commits you to

Choosing the calibrated workflow is a commitment to a process rather than to a piece of software, and it is worth being explicit about what that process is so it can be audited.

Every flight captures panel images before and after, under illumination representative of the flight, with a check that neither is shadowed or saturated. The panel’s certificate values are stored once per panel and referenced rather than retyped. The downwelling record is checked for obstruction. And a permanent ground target is flown over on every survey, giving a continuous independent check that the whole chain is working.

def calibration_readiness(flight: dict) -> list[str]:
    """Pre-flight checks for a calibrated workflow, run before take-off."""
    problems = []
    if not flight.get("panel_id"):
        problems.append("no panel assigned to this flight")
    if flight.get("panel_certificate_age_years", 0) > 3:
        problems.append("panel certificate is over three years old")
    if not flight.get("dls_present"):
        problems.append("no downwelling sensor and the sky is variable")
    if not flight.get("permanent_target_in_plan"):
        problems.append("flight plan does not cover the permanent target")
    if flight.get("exposure_mode") != "manual":
        problems.append("auto-exposure is enabled on a radiometric capture")
    return problems

Running that as a checklist item rather than a processing-time discovery is what keeps the archive complete, which is the property the whole workflow depends on.

# Verification snippet

import numpy as np


def revisit_decision(target_by_flight: dict[str, float],
                     decisions_changed: list[bool]) -> dict:
    """Re-examine the choice after a season, using what actually happened."""
    values = np.array(list(target_by_flight.values()), dtype=float)
    spread = float(values.max() - values.min()) if values.size > 1 else 0.0
    changed = sum(decisions_changed)

    if spread > 0.05:
        verdict = ("the permanent target moved by more than a crop change would — "
                   "an uncalibrated series here is not interpretable")
    elif changed:
        verdict = f"calibration changed the acted-on set on {changed} flights"
    else:
        verdict = "no flight this season needed the calibrated product"
    return {"flights": int(values.size), "target_spread": spread,
            "flights_where_it_mattered": changed, "verdict": verdict}

Revisiting the decision with a season’s evidence is more useful than making it perfectly at the start. A programme that finds calibration never changed an action has learned something worth acting on; one that finds it changed the answer three times has justified the whole apparatus.

Figure 2 — The pilot test, read as a change in decisions rather than a change in numbers.

# Writing the decision down

A decision that lives in somebody’s head is one that will be re-made differently by the next person. The output of this procedure should be a short document stored with the programme, and three sections cover it.

What was decided, and when. The workflow chosen, the date, and who made the call. A year later, the most common question is why the first four flights of a season differ from the rest, and this answers it.

The evidence. The pilot result, with its rank correlation and top-quintile agreement, and the answers to the four questions. Evidence makes the decision reviewable rather than defensible only by authority.

What would change it. A statement of the conditions under which the choice should be revisited: a new crop, a change of client, a move to time-series deliverables, or a season in which calibration changed an action. Writing the trigger down is what makes the review actually happen.

from datetime import date


def decision_record(workflow: str, pilot: dict, answers: dict,
                    *, decided_by: str) -> dict:
    """The record that makes a workflow decision reviewable later."""
    return {"workflow": workflow, "decided_on": date.today().isoformat(),
            "decided_by": decided_by, "pilot_result": pilot,
            "question_answers": answers,
            "revisit_if": ["new crop or client",
                           "deliverables become a time series",
                           "calibration changes an action in any flight",
                           "the permanent target moves by more than 0.05"]}

The revisit_if list is the part that earns its place. Most programmes never revisit the decision at all, and the ones that do usually do so after a problem rather than before.

# When to escalate

• The answer differs between crops or stages. It will. Run the pilot once per crop and record the result rather than generalising from one trial.
• A client insists on calibration for work that does not need it. Provide it; the cost is small and the argument is not worth having. Provide the pilot result too, so the decision is informed.
• The programme cannot capture the panel reliably. That is a process problem with a process fix. Until it is fixed, a consistently uncalibrated archive is more useful than a patchy calibrated one.

# Related

• Calibrated vs Uncalibrated Multispectral Workflows
• Comparing Index Values Across Flights and Sensors
• Applying Reflectance Panel Calibration in Python

← Calibrated vs Uncalibrated Multispectral Workflows