Camera Calibration and Lens Models in Python

A reconstruction can be internally perfect and geometrically wrong. Every image aligns, every tie point reprojects to a fraction of a pixel, and the resulting surface is bowed upward in the middle by forty centimetres over a two-hundred-metre site. The bundle adjustment found a self-consistent solution in which a slightly wrong lens model is compensated by a slightly wrong terrain — and with nadir imagery over flat ground, those two errors are almost indistinguishable.

Camera calibration is where that failure is prevented or created. This page covers the lens models in use, what each parameter does, the decision to fix or optimise the intrinsics, and the flight and control patterns that make a self-calibration safe. It underpins everything in automated image alignment and feature matching workflows.

Audience and prerequisites. Python 3.10+, OpenCV, and a survey with either a calibration flight or well-distributed ground control. The failure this page addresses is invisible without one of the two.

# Prerequisites

# Conceptual architecture

A lens model maps a direction in space to a pixel. The standard for drone cameras is the Brown–Conrady model: a pinhole projection with focal length and principal point, corrected by radial terms that describe barrel or pincushion distortion and tangential terms that describe a lens not quite parallel to the sensor.

Three radial coefficients and two tangential ones cover almost every rectilinear drone lens. A fisheye needs a different model entirely, because the projection is not a pinhole and fitting radial corrections to it produces coefficients that fit the calibration images and diverge outside them.

The parameter that matters most for survey accuracy is the first radial coefficient, and the reason is geometric: a small error in it changes the apparent scale toward the frame edges, which the bundle adjustment can absorb by tilting the surface. The interaction is strongest when every image looks straight down from the same height — which is exactly a standard mapping flight.

Figure 1 — The failure the whole page exists to prevent, and the flight pattern that prevents it.

# Step 1: Calibrate from a dedicated flight

A calibration flight is a short pattern over a textured area at several altitudes with a cross and obliques. Its purpose is to make the intrinsics observable — to fly the geometry that the mapping flight deliberately does not.

import cv2
import numpy as np


def calibrate_from_points(object_points: list[np.ndarray],
                          image_points: list[np.ndarray],
                          image_size: tuple[int, int],
                          *, fix_tangential: bool = False) -> dict:
    """Solve Brown–Conrady intrinsics from correspondences.

    Fixing the tangential terms is usually right for a modern drone camera:
    sensor-lens alignment is good enough that they solve to near zero, and
    leaving them free lets them absorb error that belongs elsewhere.
    """
    flags = cv2.CALIB_RATIONAL_MODEL
    if fix_tangential:
        flags |= cv2.CALIB_ZERO_TANGENT_DIST

    rms, K, dist, rvecs, tvecs = cv2.calibrateCamera(
        object_points, image_points, image_size, None, None, flags=flags,
        criteria=(cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 100, 1e-6))

    return {"rms_px": float(rms),
            "focal_px": (float(K[0, 0]), float(K[1, 1])),
            "principal_point": (float(K[0, 2]), float(K[1, 2])),
            "k": [float(d) for d in dist.ravel()[:3]],
            "p": [float(d) for d in dist.ravel()[3:5]],
            "K": K, "dist": dist, "views": len(object_points)}

An RMS above about one pixel means the model does not fit the lens, which on a rectilinear camera usually means the correspondences are poor and on a wide lens may mean the wrong model. The fisheye case is covered in fitting a Brown–Conrady model from a calibration flight.

# Step 2: Decide whether to fix the intrinsics

Three positions are defensible and the choice depends on the survey’s geometry and control.

Fix the intrinsics from a prior calibration when the mapping flight is nadir-only with weak control. The lens is not observable in that geometry, so letting the solver adjust it invites the dome.

Optimise them when the flight includes obliques, varied altitude or a cross pattern, and when good control exists. The self-calibration is then constrained and will beat a laboratory calibration, because it reflects the lens as flown — temperature, vibration and all.

Optimise a subset — focal length and the first radial term, with the rest fixed — as the middle position. It captures the parameters that genuinely drift while denying the solver the freedom to absorb a surface error.

def intrinsics_policy(flight: dict, control: dict) -> dict:
    """Which intrinsics to free, given the survey's geometry and control."""
    has_obliques = flight.get("oblique_fraction", 0.0) > 0.05
    varied_altitude = flight.get("altitude_range_m", 0.0) > 0.15 * flight.get("altitude_m", 1)
    good_control = control.get("gcp_count", 0) >= 5 and control.get("well_distributed", False)

    if has_obliques and good_control:
        return {"policy": "optimise all", "reason": "geometry and control constrain the model"}
    if (has_obliques or varied_altitude) and control.get("gcp_count", 0) >= 3:
        return {"policy": "optimise focal and k1",
                "reason": "partial constraint; free only the parameters that drift"}
    return {"policy": "fix from prior calibration",
            "reason": "nadir-only with weak control — self-calibration will bend the surface"}

Encoding the policy rather than leaving it to a default is the point. Most reconstruction software self-calibrates by default, which is right for the geometry it was designed around and wrong for a standard nadir mapping flight with three control points.

# Step 3: Detect the dome before delivering anything

import numpy as np


def detect_dome(x: np.ndarray, y: np.ndarray, residual_z: np.ndarray) -> dict:
    """Fit a quadratic surface to checkpoint residuals and report its curvature.

    A dome is a second-order term in the vertical residual across the site.
    Fitting it explicitly turns "the surface looks bowed" into a number in
    metres that can be compared against a tolerance.
    """
    x0, y0 = x.mean(), y.mean()
    dx, dy = x - x0, y - y0
    A = np.column_stack([dx ** 2 + dy ** 2, dx, dy, np.ones_like(dx)])
    coeffs, *_ = np.linalg.lstsq(A, residual_z, rcond=None)
    curvature = float(coeffs[0])

    span = float(np.hypot(dx, dy).max())
    sag = curvature * span ** 2
    resid = residual_z - A @ coeffs
    return {"curvature_per_m2": curvature,
            "sag_over_site_m": float(sag),
            "residual_after_fit_m": float(np.std(resid, ddof=1)),
            "domed": abs(sag) > 0.05,
            "note": ("a systematic dome or bowl is present — check the lens model"
                     if abs(sag) > 0.05 else "no significant curvature in the residuals")}

Checkpoints are what make this possible, and a survey without them cannot detect a dome at all — the reconstruction’s own residuals are, as the figure shows, perfectly happy. The checkpoint discipline is in checkpoint-based accuracy validation in Python.

# Step 4: Fly a calibration pattern that makes the model observable

A calibration flight and a mapping flight want opposite things. A mapping flight wants uniform coverage at constant altitude with the camera pointing down; a calibration flight wants the opposite of all three, because the parameters are only separable when the geometry varies.

Four elements make a calibration pattern work, and none of them takes long.

Two altitudes. Flying the same area at, say, 60 m and 100 m separates focal length from distance, which at a single altitude are confounded.

A cross. Lines in two perpendicular directions break the correlation between the principal point and a systematic tilt.

Obliques. A short orbit at 45° is what constrains the radial terms, because an oblique frame sees the ground at a range of distances within one image.

A textured target area. Not a checkerboard on the ground — at survey altitude a printed target is a few pixels. Textured ground with strong features in the frame corners is what the solver needs, because corner coverage is where the distortion terms are determined.

def calibration_flight_plan(centre: tuple[float, float], *, base_altitude_m: float,
                            site_extent_m: float = 150.0) -> list[dict]:
    """A short pattern that makes the intrinsics observable.

    About eight minutes of flying, once per camera per season, which is far
    cheaper than discovering a domed survey after delivery.
    """
    return [
        {"type": "grid", "altitude_m": base_altitude_m, "heading_deg": 0,
         "extent_m": site_extent_m, "overlap": 0.8},
        {"type": "grid", "altitude_m": base_altitude_m, "heading_deg": 90,
         "extent_m": site_extent_m, "overlap": 0.8},
        {"type": "grid", "altitude_m": base_altitude_m * 1.7, "heading_deg": 45,
         "extent_m": site_extent_m, "overlap": 0.75},
        {"type": "orbit", "altitude_m": base_altitude_m, "pitch_deg": -45,
         "radius_m": site_extent_m / 2, "images": 24},
    ]

# Step 5: Store the calibration with the camera, not with the job

A calibration belongs to a physical camera and should be stored against its serial number, with the date and the conditions it was measured in. Two conventions make that useful rather than decorative.

Record the temperature if it is available. Focal length changes measurably with temperature on the small lenses drone cameras use, and a calibration measured at 5 °C applied to a survey flown at 30 °C carries a small systematic error.

Record the flight the calibration came from, so it can be re-derived. A calibration whose provenance is a spreadsheet cell is one nobody will trust in a year.

from datetime import date


def store_calibration(serial: str, calibration: dict, *, source_flight: str,
                      temperature_c: float | None) -> dict:
    """A calibration record tied to the physical camera."""
    return {"camera_serial": serial, "measured_on": date.today().isoformat(),
            "source_flight": source_flight, "temperature_c": temperature_c,
            "focal_px": calibration["focal_px"],
            "principal_point": calibration["principal_point"],
            "k": calibration["k"], "p": calibration["p"],
            "rms_px": calibration["rms_px"], "views": calibration["views"]}


def calibration_age_warning(record: dict, *, max_age_days: int = 180) -> str | None:
    """Prompt a re-calibration before the stored one is too old to trust."""
    from datetime import datetime
    age = (datetime.now().date() - date.fromisoformat(record["measured_on"])).days
    if age > max_age_days:
        return (f"calibration for {record['camera_serial']} is {age} days old — "
                "re-fly the calibration pattern before relying on fixed intrinsics")
    return None

The age warning matters because a fixed-intrinsics policy is only as good as the calibration behind it. A camera that has been transported, serviced or simply used for six months has moved, and a stale calibration applied as fixed is worse than a constrained self-calibration.

# Why ground control is not a substitute

A common position is that enough ground control fixes any lens error, so calibration does not matter. It is half right in a way that is worth unpicking.

Control points constrain the reconstruction at the points themselves. A domed surface with control at five points is pulled down to the truth at those five locations and remains domed between them — the deformation redistributes rather than disappearing. With control only around the perimeter, which is the usual layout, the dome is anchored at the edges and free in the middle, which is the worst arrangement.

Control distributed across the interior does suppress doming substantially, and it is the reason a well-controlled survey can self-calibrate safely. But it takes more points than most projects place, and those points cost field time that a calibration flight does not.

The practical hierarchy is: fly a geometry that constrains the lens, calibrate the camera and store it, place control across the interior as well as the perimeter, and keep checkpoints to detect what the first three missed. Each step is cheap and each one catches a different failure, which is why doing one of them and skipping the rest is how domed surveys get delivered.

# Parameter deep-dive

# Verification and output inspection

import numpy as np


def compare_calibrations(a: dict, b: dict, image_width: int) -> dict:
    """Do two calibrations of the same camera agree?

    Focal length is compared as a fraction of image width so the number is
    interpretable, and k1 in absolute terms because its scale is fixed.
    """
    df = abs(a["focal_px"][0] - b["focal_px"][0]) / image_width
    dk1 = abs(a["k"][0] - b["k"][0])
    dpp = float(np.hypot(a["principal_point"][0] - b["principal_point"][0],
                         a["principal_point"][1] - b["principal_point"][1]))

    problems = []
    if df > 0.005:
        problems.append(f"focal length differs by {df:.3%} of image width")
    if dk1 > 0.01:
        problems.append(f"k1 differs by {dk1:.4f}")
    if dpp > 0.01 * image_width:
        problems.append(f"principal point differs by {dpp:.0f} px")
    return {"focal_delta_fraction": df, "k1_delta": dk1,
            "principal_point_delta_px": dpp, "problems": problems,
            "consistent": not problems}

Two calibrations of the same camera that disagree materially mean one of them was poorly constrained, and comparing a fresh calibration against the stored one before every season is a cheap way to catch a lens that has been knocked.

Figure 3 — Four groups, very unequal in how much they matter.

Figure 4 — The trade is twenty minutes against a parameter nothing else constrains.

# Troubleshooting

The surface is domed and the residuals are excellent. Self-calibration absorbed a lens error. Fix the intrinsics from a prior calibration, or re-fly with obliques.

The calibration RMS is above a pixel. Poor correspondences, too few views, or the wrong model for a wide lens. Check the number of views before the model: a solve from a handful of images is unconstrained whatever the lens.

k3 solves to a large value. Over-parameterisation. Fix it at zero unless the lens genuinely needs it.

Two flights of the same camera give different intrinsics. Expected to a degree — focal length drifts with temperature — but a large difference means one flight did not constrain the model.

Undistorted images look worse at the edges. The coefficients are being extrapolated beyond the calibration’s coverage. Calibrate with points across the whole frame, including the corners, because the distortion terms are determined almost entirely by what happens away from the centre.

A fisheye lens produces divergent corrections. The wrong model. Use the fisheye solver, not radial terms on a pinhole projection; radial terms fitted to a fisheye match the calibration images closely and diverge outside their coverage.

The reconstruction rejects the supplied intrinsics and solves its own. Most engines treat a supplied calibration as an initial estimate unless told otherwise. Setting the parameters as fixed is a separate option from supplying them, and forgetting it produces a self-calibration with a good starting point rather than a fixed one.

# Related

• Fitting a Brown–Conrady Model from a Calibration Flight
• Deciding When to Fix Versus Optimise Intrinsics
• Handling Rolling Shutter in Fast Flight Lines
• Diagnosing Doming and Bowl Effect in Reconstructions
• Optimizing Bundle Adjustment with Python

← Automated Image Alignment & Feature Matching Workflows