Resolving Panel Detection Failures in Calibration

The calibration run stops with “no panel found in frame 0004”, or — worse — completes, having found something. On inspection the detected region is a patch of bright gravel behind the operator, and every reflectance value in the survey has been scaled by its radiance.

Panel detection is a small step with outsized consequences, because its output is a single number that multiplies the entire dataset. This page covers making it robust, giving it a manual fallback, and checking that what it found is actually the panel. It is part of the diagnostic sequence in troubleshooting multispectral and thermal failures, and it feeds the measurement described in applying reflectance panel calibration in Python.

Why brightness alone is a poor detector

The standard approach thresholds on brightness and takes the largest bright region. It fails in both directions.

It misses the panel when the panel is not the brightest thing in frame. A grey panel at 0.5 reflectance photographed next to white gravel, a light-coloured vehicle, or a patch of sky in a tilted frame is not the brightest region, and a percentile threshold tuned for one site fails at the next.

It finds the wrong thing when something brighter is present. Concrete, painted lines, a reflective vest, the sky through a gap — all of them pass a brightness test and none of them has a known reflectance.

What distinguishes a panel is not its brightness but its geometry and uniformity: it is a quadrilateral of near-constant value, with sharp edges, occupying a substantial and predictable fraction of the frame. Detecting on those properties is both more reliable and easier to validate.

Brightness against geometric detection on a difficult panel frame Two views of the same panel image, which contains a grey panel, a patch of bright gravel and a white vehicle. Brightness thresholding selects the vehicle, which is brightest, and misses the panel entirely. Geometric detection selects the panel by its quadrilateral shape, uniform interior and expected size, and rejects the gravel as non-uniform and the vehicle as the wrong shape. A note records that the brightness result would scale every reflectance in the survey by the vehicle's radiance. brightness threshold geometric detection panel gravel vehicle — selected panel — selected rejected: not uniform rejected: wrong shape The brightness result would scale every reflectance in the survey by the vehicle. A panel is not the brightest thing in frame; it is the most uniform quadrilateral of about the right size.

Figure 1 — Why the obvious detector is the wrong one, on a frame that is not unusual.

Minimal reproducible solution

import cv2
import numpy as np


def detect_panel(frame: np.ndarray, *, min_area_fraction: float = 0.02,
                 max_area_fraction: float = 0.5,
                 max_interior_cv: float = 0.06) -> dict:
    """Locate a reflectance panel by geometry and uniformity, not brightness.

    Three filters in sequence: the region must be a convex quadrilateral, it
    must occupy a plausible share of the frame, and its interior must be
    uniform. Gravel fails the third, a vehicle the first, and the sky the
    second — none of which a brightness test can distinguish.
    """
    img = frame.astype(np.float32)
    norm = np.clip((img - np.percentile(img, 1)) /
                   max(np.percentile(img, 99) - np.percentile(img, 1), 1e-6) * 255,
                   0, 255).astype(np.uint8)
    blurred = cv2.GaussianBlur(norm, (7, 7), 0)
    edges = cv2.Canny(blurred, 40, 120)
    edges = cv2.dilate(edges, np.ones((3, 3), np.uint8), iterations=1)

    contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    frame_area = img.size
    candidates = []
    for c in contours:
        area = cv2.contourArea(c)
        if not (min_area_fraction * frame_area < area < max_area_fraction * frame_area):
            continue
        approx = cv2.approxPolyDP(c, 0.03 * cv2.arcLength(c, True), True)
        if len(approx) != 4 or not cv2.isContourConvex(approx):
            continue

        mask = np.zeros(img.shape, np.uint8)
        cv2.drawContours(mask, [approx], -1, 1, cv2.FILLED)
        interior = cv2.erode(mask, np.ones((21, 21), np.uint8)).astype(bool)
        if interior.sum() < 500:
            continue
        values = img[interior]
        cv = float(values.std() / max(values.mean(), 1e-6))
        if cv > max_interior_cv:
            continue
        candidates.append({"corners": approx.reshape(-1, 2), "area": float(area),
                           "interior_cv": cv, "median": float(np.median(values)),
                           "pixels": int(interior.sum())})

    if not candidates:
        return {"found": False,
                "note": "no uniform quadrilateral of plausible size — supply corners manually"}
    best = min(candidates, key=lambda c: c["interior_cv"])
    return {"found": True, **best, "candidates": len(candidates)}

Selecting the most uniform candidate rather than the largest or brightest is the choice that makes this robust. A panel is manufactured to be uniform; almost nothing else in a field scene is.

The manual fallback

Automatic detection will fail on some frames, and a pipeline that stops there costs a whole flight. A manual fallback — four corner coordinates supplied once per flight — keeps the run going.

import numpy as np


def measure_from_corners(frame: np.ndarray, corners: np.ndarray,
                         *, erode_px: int = 15,
                         keep_percentile: tuple = (20, 80)) -> dict:
    """Measure a panel from manually supplied corners, with the same guards."""
    import cv2
    mask = np.zeros(frame.shape, np.uint8)
    cv2.fillConvexPoly(mask, corners.astype(np.int32), 1)
    interior = cv2.erode(mask, np.ones((erode_px * 2 + 1,) * 2, np.uint8)).astype(bool)

    values = frame[interior]
    if values.size < 500:
        raise ValueError("too few interior pixels; check the supplied corners")
    lo, hi = np.percentile(values, keep_percentile)
    core = values[(values >= lo) & (values <= hi)]
    return {"radiance": float(np.median(core)), "pixels": int(core.size),
            "interior_cv": float(core.std() / max(core.mean(), 1e-6)),
            "source": "manual corners"}

Recording source distinguishes a frame measured automatically from one measured by hand, which matters when a calibration is later questioned.

Why automatic panel detection fails, checked in order A four-stage check. Stage one confirms the panel is actually in the frames being searched, since a panel imaged before the camera began logging is absent from the dataset entirely. Stage two confirms it is not saturated, because a detector keyed on a bright uniform region finds nothing when that region has clipped to a single value. Stage three confirms the panel's target or marker is legible at the flight's ground sample distance, which fails when the panel was imaged from too high. Stage four confirms the search is running on raw rather than already-corrected imagery. 1. present at all imaged before logging began means absent 2. not saturated a clipped region is not a bright uniform one 3. marker legible fails when imaged from too high 4. raw imagery not already-corrected frames Three of the four are flight-procedure problems, and all three are cheap to prevent.

Figure 3 — Four checks, and detection tuning helps with only one of them.

Edge-case matrix

Situation Symptom Handling
Panel not the brightest object Detector picks something else Geometric detection
Panel too small in frame Rejected on area Re-shoot closer, or lower the minimum
Panel partly out of frame Not a quadrilateral Reject; use the other panel capture
Panel in shadow Found, but low radiance Uniformity passes; the reconciliation check catches it
Strong specular glint Interior uniformity fails Percentile window in the measurement
Panel on a textured surface Edges unclear Place it on a plain background
Multiple panels in frame Several candidates Take the most uniform, log the count
Panel saturated Uniform and clipped Explicit saturation check

The saturated case deserves its own guard, because a clipped panel is perfectly uniform and passes every geometric test:

import numpy as np


def saturation_guard(values: np.ndarray, *, ceiling: int = 65000,
                     max_fraction: float = 0.001) -> None:
    """Reject a panel measurement containing clipped pixels."""
    frac = float(np.count_nonzero(values >= ceiling) / max(values.size, 1))
    if frac > max_fraction:
        raise ValueError(f"{frac:.1%} of the panel interior is saturated — "
                         "re-shoot at a lower exposure; this measurement is unusable")

Verification snippet

import numpy as np


def validate_detection(measurement: dict, *, expected_reflectance: float,
                       frame_median: float) -> dict:
    """Is the detected region plausibly the panel we think it is?"""
    problems = []
    ratio = measurement["radiance"] / max(frame_median, 1e-6)

    if measurement["interior_cv"] > 0.06:
        problems.append(f"interior variation {measurement['interior_cv']:.3f} is high "
                        "for a manufactured panel")
    if ratio < 1.2:
        problems.append("the detected region is barely brighter than the scene — "
                        "it may not be the panel")
    if ratio > 12:
        problems.append("the detected region is extremely bright — possibly the sky "
                        "or a specular surface")
    if measurement["pixels"] < 2000:
        problems.append(f"only {measurement['pixels']} usable interior pixels")

    return {"brightness_ratio": float(ratio), "problems": problems,
            "usable": not problems,
            "expected_reflectance": expected_reflectance}

The brightness ratio is a sanity check rather than a detector, and that distinction is the point: brightness is poor at finding a panel and perfectly good at confirming that a geometrically detected region is plausible.

Interior uniformity separates a panel from everything else in frame Coefficient of variation of the interior for five candidate regions in a panel frame. The reflectance panel is at zero point zero two, well below the acceptance threshold of zero point zero six. A concrete apron is at zero point zero nine, a vehicle bonnet at zero point one four, gravel at zero point two one, and grass at zero point three one. A note records that a saturated panel would also read near zero, which is why the saturation guard is separate from the uniformity test. reflectance panel 0.02 concrete apron 0.09 vehicle bonnet 0.14 gravel 0.21 grass 0.31 acceptance threshold 0.06 A saturated panel also reads near zero here, which is why saturation is a separate guard.

Figure 2 — One number separates the panel from every other candidate in a typical frame.

Making detection unnecessary

The most reliable panel detection is the one that has almost nothing to do. Three capture habits reduce the problem to a formality.

Flag the panel frames at capture. Most rigs record a marker, or the operator can note the frame numbers. Searching two known frames instead of a whole flight removes every false positive from the rest of the survey at a stroke.

Fill a good share of the frame. A panel occupying two percent of the image has a small interior after erosion and is easily confused; one occupying a fifth is unmistakable to any detector. Moving closer costs nothing.

Use a plain background. A panel placed on short grass or a clean apron has sharp, complete edges; one placed on gravel or in long vegetation has broken edges that defeat contour detection. A folded groundsheet solves it permanently.

None of these needs new equipment, and together they turn panel detection from a recurring failure into a step that has never failed.

When to escalate

  • Detection fails on every frame of a flight. The panel was probably not photographed, or was photographed too small to use. Check the capture before adjusting thresholds.
  • A detected panel passes every check and the reflectance still comes out wrong. Compare the two panel captures against each other; a shadowed one passes uniformity and reconciliation catches it.
  • The site has a large uniform bright surface. A white membrane roof can beat a panel on every geometric test. Constrain the search to frames flagged as panel captures rather than searching the whole flight.

← Troubleshooting Multispectral and Thermal Failures