Colour Balancing Images Before Mosaicking

Correcting a mosaic after it exists is fighting the wrong battle. Once two images have been cut together, the information needed to separate “this area is genuinely darker” from “this image was underexposed” has already been discarded. The correction belongs on the individual images, before the mosaic is assembled.

This page covers that pre-processing stage: normalising exposure from what the camera recorded, solving residual gains across the block, and handling the colour cast that a global brightness fix leaves behind. It is part of orthomosaic radiometry and seamline control.

# Start from what the camera recorded

The camera already knows why two frames differ in brightness, and it wrote it into the EXIF. Exposure time, aperture and ISO together determine how much light reached the sensor for a given scene radiance, and normalising by them removes the entire auto-exposure component before any statistics are computed.

The relationship is straightforward. Doubling the exposure time doubles the signal. Halving the f-number quadruples it. Doubling the ISO doubles the recorded value for the same light. A single relative factor captures all three.

import numpy as np


def exposure_factor(shutter_s: float, f_number: float, iso: float) -> float:
    """Relative sensor exposure for one frame.

    Larger means more light was collected, so the recorded pixel values should
    be divided by this factor to express every frame on a common scale.
    """
    return (shutter_s * iso) / (f_number ** 2)


def normalise_by_exposure(values: np.ndarray, shutter_s: float,
                          f_number: float, iso: float,
                          *, reference: float) -> np.ndarray:
    """Rescale one image's pixels onto the reference frame's exposure."""
    factor = exposure_factor(shutter_s, f_number, iso)
    return values.astype("float32") * (reference / factor)

On a survey flown with exposure locked, this step does nothing and costs nothing. On a survey flown on auto — which is most of them — it removes the largest single source of frame-to-frame variation before anything harder is attempted.

Figure 1 — Removing the auto-exposure component costs nothing and removes most of the variation.

# Minimal reproducible solution

With exposure normalised, the remaining differences are illumination and atmosphere, and they are solved across the block rather than per image.

from itertools import combinations
import rasterio


def overlap_ratio(path_a: str, path_b: str, *, band: int = 1,
                  min_pixels: int = 5000) -> float | None:
    """Mean brightness ratio over the pixels two orthorectified frames share.

    Returns None when the shared area is too small for the ratio to mean
    anything — a ratio computed over a few hundred pixels is noise, and it will
    drag a least-squares solve toward nonsense.
    """
    with rasterio.open(path_a) as sa, rasterio.open(path_b) as sb:
        a = sa.read(band, masked=True).astype("float32")
        b = sb.read(band, masked=True).astype("float32")

    shared = ~a.mask & ~b.mask
    if shared.sum() < min_pixels:
        return None

    va, vb = a.data[shared], b.data[shared]
    # Trim the extreme deciles: tall objects and vehicles live there, and they
    # bias a mean ratio far more than their pixel count suggests.
    d = va - vb
    lo, hi = np.percentile(d, [10, 90])
    keep = (d >= lo) & (d <= hi)
    if keep.sum() < min_pixels // 2:
        return None

    mb = float(vb[keep].mean())
    return float(va[keep].mean()) / mb if mb > 0 else None

Feeding those ratios to the block solve from the topic page gives one gain per image, consistent across the whole survey rather than chained pairwise down a flight line.

# Gain, offset, and why the choice matters

A multiplicative gain and an additive offset correct different physical effects, and applying the wrong one produces a mosaic that matches in some tonal ranges and diverges in others.

Gain scales the signal. It is the right model for exposure and for sensitivity differences, because both scale the light reaching the sensor. A gain preserves contrast ratios — a shadow that was half as bright as its surroundings stays half as bright.

Offset adds a constant. It is the right model for path radiance: haze, atmospheric scattering and internal lens flare all add light that was never reflected from the ground. An offset does not preserve contrast ratios, and applying one where a gain belonged flattens shadows visibly.

The diagnostic is the shadows. If two images agree in the midtones after a gain correction but their dark areas still differ, there is an additive component that a gain cannot reach.

def fit_gain_and_offset(values_a: np.ndarray, values_b: np.ndarray) -> dict:
    """Fit a = gain * b + offset over shared pixels, and say which matters.

    A significant offset relative to the image's dynamic range means haze or
    flare; a gain far from one with a negligible offset means exposure.
    """
    gain, offset = np.polyfit(values_b.astype("float64"),
                              values_a.astype("float64"), 1)
    dynamic_range = float(values_a.max() - values_a.min()) or 1.0
    return {
        "gain": float(gain),
        "offset": float(offset),
        "offset_fraction": abs(float(offset)) / dynamic_range,
        # Above a few per cent of the range, the additive term is doing real
        # work and a gain-only correction will leave the shadows mismatched.
        "needs_offset": abs(float(offset)) / dynamic_range > 0.03,
    }

# Per-band balance and colour cast

Applying one gain to all three bands corrects brightness and leaves colour alone, which is usually what you want. Solving a separate gain per band corrects colour too — and will happily invent a colour shift that was not there, because a genuinely green field and a green colour cast look identical to a per-band statistic.

The safe rule is to solve per band only where the survey contains something with a known, stable colour across the whole site — a calibration panel, a concrete apron, a road surface. Without such a reference, solve one gain per image on luminance and accept the cast.

Figure 2 — Per-band solving needs an anchor or it will manufacture one.

Figure 3 — Four stages, ending in a check with two conditions.

# Edge-case matrix

# Verification snippet

def balance_report(before: dict, after: dict) -> dict:
    """Did balancing actually help, and did it cost anything?

    Two failure modes look like success on a spread statistic alone: clipping,
    which reduces spread by destroying highlight detail, and over-correction,
    which flattens real scene variation along with the artefacts.
    """
    improvement = (before["spread_pct"] - after["spread_pct"]) / before["spread_pct"]
    return {
        "spread_before_pct": before["spread_pct"],
        "spread_after_pct": after["spread_pct"],
        "improvement": float(improvement),
        "clipped_fraction": after.get("clipped_fraction", 0.0),
        "ok": improvement > 0.3 and after.get("clipped_fraction", 0.0) < 0.001,
    }

Both conditions matter. A run that halves the spread while clipping a per cent of the highlights has made the mosaic look more uniform and made the data worse.

Keep the before-and-after figures in the job’s record rather than discarding them once the mosaic looks acceptable. Over a season they answer a question that is otherwise guesswork: whether the balancing step is doing real work on every flight, or whether a particular aircraft and camera combination produces imagery so consistent that the whole stage could be skipped. Sites flown with exposure locked under stable overcast frequently fall into the second category, and knowing that turns an unexamined habit into a deliberate choice.

# When to escalate

• Spread barely improves. The variation is probably within-frame — vignetting or hotspot — which no per-image gain can reach.
• Balancing clips regardless of the cap. The source images are already near saturation. This is a flight problem, not a processing one.
• The site genuinely changes colour across itself. A survey spanning a harvest boundary or a wet and dry area will fight any global balance. Balance on luminance and leave colour alone.

# Related

• Orthomosaic Radiometry and Seamline Control
• Fixing Visible Seamlines and Colour Banding
• Radiometric Calibration in Python

← Orthomosaic Radiometry and Seamline Control