Fitting a Brown–Conrady Model from a Calibration Flight

The classic camera calibration uses a printed checkerboard held at various angles a metre from the lens. At survey altitude that is useless: a target large enough to fill the frame at 80 m would be the size of a tennis court, and a camera focused at infinity behaves differently from one focused at a metre anyway.

Calibrating a survey camera therefore means solving the intrinsics from aerial imagery, using tie points from a reconstruction as the correspondences. This page covers doing that, choosing which parameters to solve for, and — most importantly — recognising when the result is unconstrained rather than wrong. It supports the policy decisions in camera calibration and lens models in Python.

What makes an aerial calibration different

The correspondences are tie points, not targets. They come from the matching stage and carry its errors, which means a calibration is only as good as the reconstruction that produced its input.

Scale comes from outside. A checkerboard has known dimensions; a landscape does not. Without ground control or accurate camera positions, the whole solution is scale-free and the focal length is undetermined.

Depth variation is small. A flat site at constant altitude gives every point almost the same depth, which is the geometry in which focal length and camera height are least separable.

All three point the same way: the calibration flight’s job is to introduce the variation that a mapping flight lacks, and without that variation the solve is ill-conditioned regardless of how many images it has.

Parameter uncertainty against flight geometry Four flight geometries compared by the uncertainty they leave in the calibration parameters. A single-altitude nadir grid leaves focal length very poorly determined and the first radial term poorly determined. Adding a second altitude improves focal length substantially. Adding a cross pattern improves the principal point. Adding an oblique orbit improves the radial terms, leaving all four parameters well determined. A note states that each element addresses a different parameter, which is why the full pattern is needed rather than more images of the same geometry. focal principal pt k1 k2 nadir grid, one altitude + second altitude + cross pattern + oblique orbit poorpoor weakpoor goodpoor weakpoor goodgood weakweak goodgood goodgood Each element addresses a different parameter. Which is why more images of the same geometry never fix an ill-conditioned solve.

Figure 1 — Why the calibration pattern has four parts rather than one.

Minimal reproducible solution

import cv2
import numpy as np


def calibrate_from_reconstruction(object_points: list[np.ndarray],
                                  image_points: list[np.ndarray],
                                  image_size: tuple[int, int],
                                  *, solve_k3: bool = False,
                                  solve_tangential: bool = False) -> dict:
    """Solve Brown–Conrady intrinsics from reconstruction tie points.

    Parameters are disabled rather than solved by default. Every free
    parameter is one more thing that can absorb error from the others, and on
    a modern drone lens k3 and the tangential terms genuinely are near zero.
    """
    flags = 0
    if not solve_k3:
        flags |= cv2.CALIB_FIX_K3
    if not solve_tangential:
        flags |= cv2.CALIB_ZERO_TANGENT_DIST

    rms, K, dist, rvecs, tvecs = cv2.calibrateCamera(
        object_points, image_points, image_size, None, None, flags=flags,
        criteria=(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 200, 1e-7))

    per_view = []
    for i, (obj, img) in enumerate(zip(object_points, image_points)):
        projected, _ = cv2.projectPoints(obj, rvecs[i], tvecs[i], K, dist)
        err = np.linalg.norm(projected.reshape(-1, 2) - img.reshape(-1, 2), axis=1)
        per_view.append(float(np.mean(err)))

    return {"rms_px": float(rms), "K": K, "dist": dist,
            "focal_px": (float(K[0, 0]), float(K[1, 1])),
            "principal_point": (float(K[0, 2]), float(K[1, 2])),
            "k": [float(v) for v in dist.ravel()[:3]],
            "per_view_error_px": per_view,
            "views": len(object_points)}

Disabling parameters by default rather than solving everything is the choice that most improves a calibration’s stability. A solve with every term free will always report a lower RMS, because it has more freedom to fit; whether the extra terms describe the lens or the noise is a different question, and on aerial correspondences it is usually the noise.

Judging whether the solve is constrained

import cv2
import numpy as np


def parameter_uncertainty(object_points, image_points, image_size,
                          *, flags: int = 0) -> dict:
    """Standard deviations of the solved intrinsics, which OpenCV can report.

    A calibration with a low RMS and a huge focal-length uncertainty is not a
    good calibration; it is an underdetermined one that happened to fit. The
    uncertainties are what distinguish the two, and they are free.
    """
    rms, K, dist, _, _, std_int, _, _ = cv2.calibrateCameraExtended(
        object_points, image_points, image_size, None, None, flags=flags)

    labels = ["fx", "fy", "cx", "cy", "k1", "k2", "p1", "p2", "k3"]
    stds = {name: float(v) for name, v in zip(labels, std_int.ravel())}

    problems = []
    if stds["fx"] / float(K[0, 0]) > 0.01:
        problems.append(f"focal length is uncertain by {stds['fx'] / K[0, 0]:.1%} — "
                        "add a second altitude")
    if stds["cx"] > 0.02 * image_size[0]:
        problems.append("principal point is poorly determined — add a cross pattern")
    if stds["k1"] > 0.01:
        problems.append("k1 is poorly determined — add oblique frames")

    return {"rms_px": float(rms), "std": stds, "problems": problems,
            "constrained": not problems}

Reporting the uncertainties alongside the values is the practice that distinguishes a usable calibration from a plausible one, and it costs one different function call.

The calibration fit from capture to a usable parameter set A five-stage sequence. Stage one captures a pattern or a well-featured scene from convergent viewpoints at varied roll and distance. Stage two detects correspondences to sub-pixel accuracy, since the fit cannot be better than the measurements feeding it. Stage three solves for intrinsics and poses together, because they are correlated and solving either alone biases both. Stage four examines the residual field for structure, which reveals an unmodelled effect that a lower total residual would hide. Stage five records the result against the camera serial and the date, since a calibration without provenance cannot be applied confidently later. 1. capture convergent views, varied roll and range 2. detect sub-pixel correspondences 3. solve intrinsics and poses together, not apart 4. inspect residual field for structure, not just size 5. record against camera serial and date Stage 4 is the one most often skipped, and the one that finds an unmodelled effect.

Figure 3 — Five stages; the fourth is where a bad fit is caught.

Edge-case matrix

Situation Symptom Handling
Single altitude Focal length uncertain Add a second altitude
Nadir only k1 uncertain Add obliques
One flight direction Principal point uncertain Add a cross
Few tie points near the corners Distortion extrapolated Choose views with corner coverage
All terms solved Low RMS, unstable values Fix k3 and tangential
No control or camera positions Scale-free, focal undetermined Supply either
Tie points from a poor reconstruction Calibration inherits the errors Fix the matching first
Fisheye lens Divergent radial terms Use the fisheye model

The corner-coverage row deserves attention because it is easy to satisfy and easy to miss. Distortion is determined almost entirely by points far from the image centre, so a set of views whose tie points cluster in the middle produces a distortion model that is extrapolated everywhere it matters.

import numpy as np


def corner_coverage(image_points: list[np.ndarray],
                    image_size: tuple[int, int]) -> dict:
    """What share of the frame's outer region carries correspondences."""
    w, h = image_size
    cx, cy = w / 2, h / 2
    max_r = float(np.hypot(cx, cy))

    radii = []
    for pts in image_points:
        p = pts.reshape(-1, 2)
        radii.extend(np.hypot(p[:, 0] - cx, p[:, 1] - cy) / max_r)

    r = np.asarray(radii)
    outer = float((r > 0.7).mean())
    return {"points": int(r.size), "outer_region_fraction": outer,
            "adequate": outer > 0.15,
            "note": ("corner coverage is adequate" if outer > 0.15 else
                     "few correspondences beyond 70 % of the frame radius — "
                     "the distortion model is extrapolated at the edges")}

Verification snippet

import cv2
import numpy as np


def holdout_check(object_points, image_points, image_size,
                  *, holdout: int = 5) -> dict:
    """Calibrate on most views and test on the rest.

    A calibration that fits its own views and fails on held-out ones has
    fitted noise, which is exactly what over-parameterisation produces and
    exactly what an in-sample RMS cannot show.
    """
    n = len(object_points)
    if n <= holdout + 5:
        return {"note": "not enough views for a holdout"}

    train_obj, train_img = object_points[:-holdout], image_points[:-holdout]
    test_obj, test_img = object_points[-holdout:], image_points[-holdout:]

    rms, K, dist, _, _ = cv2.calibrateCamera(train_obj, train_img, image_size, None, None,
                                             flags=cv2.CALIB_FIX_K3 | cv2.CALIB_ZERO_TANGENT_DIST)
    errors = []
    for obj, img in zip(test_obj, test_img):
        ok, rvec, tvec = cv2.solvePnP(obj, img, K, dist)
        if not ok:
            continue
        proj, _ = cv2.projectPoints(obj, rvec, tvec, K, dist)
        errors.append(float(np.mean(np.linalg.norm(
            proj.reshape(-1, 2) - img.reshape(-1, 2), axis=1))))

    test_rms = float(np.mean(errors)) if errors else float("nan")
    return {"train_rms_px": float(rms), "test_rms_px": test_rms,
            "ratio": test_rms / max(float(rms), 1e-9),
            "overfitted": test_rms > 2 * float(rms)}
Training and holdout error against the number of free parameters Two curves against the number of free calibration parameters, from four to nine. The training error falls steadily from zero point nine pixels to zero point five as parameters are added. The holdout error falls to a minimum of zero point six at six parameters and then rises to one point four at nine, the classic overfitting shape. A marker at six parameters shows the best generalising model, corresponding to focal length, principal point and two radial terms. 456 79 free parameters error training error — always falls holdout error — rises past six best: f, cx, cy, k1, k2 An in-sample RMS cannot show this, which is why the holdout is worth the five views.

Figure 2 — The reason to disable parameters rather than solve them.

Getting the correspondences in the first place

The calibration above needs object points and image points, and on an aerial flight those come from a reconstruction rather than from a target. Two sources work, with different trade-offs.

Tie points from a completed reconstruction are plentiful and free. Their object coordinates come from the same solution whose intrinsics are being estimated, which sounds circular and is acceptable in practice provided the reconstruction was well controlled: the geometry constrains the solution, and re-solving the intrinsics against the resulting points refines rather than invents them.

Surveyed points visible in the imagery — control targets, or any identifiable feature whose coordinates were measured — are independent and far fewer. A handful of them scattered across the frame is worth more than thousands of tie points clustered in the middle, because they break the circularity.

The arrangement that works uses both: tie points for coverage and surveyed points as an independent check on the result, which is exactly the holdout above with better data.

When to escalate

  • The uncertainties are large despite a full calibration pattern. The tie points may be poor. Look at the reconstruction before the calibration.
  • The lens is fisheye or strongly wide-angle. Use the fisheye model; radial terms fitted to it will fit the views and diverge outside them.
  • Calibrations vary substantially between flights of the same camera. Beyond the expected thermal drift, that means the geometry is not constraining the solve. Compare the reported uncertainties across the flights.

← Camera Calibration and Lens Models in Python