Fixing OpenSfM Bundle Adjustment Divergence

Your ODM run does not crash — it finishes, and the model is wrong. The point cloud bows upward at the edges, flight lines shear apart, or the OpenSfM log shows the reprojection error climbing across iterations instead of settling, sometimes ending with a “bundle adjustment did not converge” note. This is divergence: the non-linear solver that jointly refines camera poses, intrinsics, and 3D points has walked away from the true minimum instead of toward it, and every downstream product inherits the distortion. The cause is never the solver itself — it is a bad constraint the solver faithfully honoured: over-tight GPS priors, too few valid matches to pin the geometry, a camera model that does not fit the lens, or a contaminated ground control point (GCP) dragging the solution. This page shows how to read the reprojection statistics, identify which constraint is poisoning the solve, and correct it. It is the reconstruction-quality deep dive for setting up OpenDroneMap with Python.

Why bundle adjustment diverges

Bundle adjustment minimizes the sum of squared reprojection residuals — the pixel distance between each observed feature xij\mathbf{x}_{ij} and where the current camera and point estimates project it, π(K,Ri,ti,Xj)\pi(K, R_i, t_i, X_j):

RMSE=1N∑i,j∥xij−π(K,Ri,ti,Xj)∥22\mathrm{RMSE} = \sqrt{\frac{1}{N}\sum_{i,j}\big\lVert \mathbf{x}_{ij} - \pi(K, R_i, t_i, X_j)\big\rVert_2^{2}}

A healthy solve on calibrated drone optics drives this toward roughly 0.3–0.7 normalized pixels and decreases monotonically. Divergence — an RMSE that rises or oscillates — has four dominant root causes, each a constraint the solver cannot satisfy without distorting everything else:

  1. Over-weighted GPS priors. ODM converts GPS accuracy into a prior weight; if standalone-GNSS positions (3–10 m real error) are treated as if they were RTK-tight (centimetres), the solver is forced to honour physically impossible camera positions and warps the block to fit them. This is the most common cause of edge-bowing (“doming”).
  2. Insufficient overlap or matches. When a region has too few tie points, its cameras are under-constrained; the solver has a near-degenerate sub-problem and small errors blow up into drift. Thin cross-strip overlap is the usual culprit.
  3. A wrong camera model. Forcing a simple perspective model onto a fisheye lens (or the reverse) means the intrinsics can never explain the observations, so residuals never converge and the solver oscillates trying to absorb lens distortion into pose.
  4. GCP contamination. A single mistyped GCP coordinate, a mislabeled marker, or a GCP with a wrong CRS acts as an enormously heavy, wrong constraint that the least-squares solve spreads across the whole model.

The fix is to relax or correct the offending constraint and let a robust loss suppress the residual outliers, rather than letting one bad prior dominate the squared-error cost.

A single gross outlier pulling the reconstruction apart A schematic of a small reconstruction. Six camera positions sit along a flight line, each observing a cluster of well-triangulated ground points, drawn with short residual lines. One observation is a gross mismatch: a tie point placed far below the ground surface, connected to two cameras by long residual lines. Under a squared cost this single observation contributes more than all the others combined, so the optimiser tilts the entire camera track to reduce it, shown by a ghosted set of displaced camera positions. The caption notes the cost is quadratic in the residual, so one observation at fifty pixels outweighs a thousand at one pixel. camera track (solved) tilted by one bad tie well-triangulated points — residuals under 1 px one mismatched tie point, ≈ 50 px residual Under a squared cost, 50² = 2500 — more than a thousand one-pixel observations combined. The solver is behaving correctly. The cost function is the thing that is wrong.

Figure 1 — Divergence rarely means the optimiser is broken. It usually means one observation is being allowed to speak louder than every good observation put together, which is a property of the loss, not of the data.

Minimal reproducible solution

The wrapper below reads the reprojection RMSE from OpenSfM’s own statistics, and when it exceeds a convergence threshold re-runs ODM with corrected constraints: a GPS accuracy that matches how the positions were actually captured, an explicit camera model, and — if a control file exists — GCPs seeded so the datum is pinned by survey truth rather than weak GNSS. Matching --gps-accuracy to reality is the single highest-leverage change.

import json
import subprocess
import logging
from pathlib import Path

logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")

def read_reprojection_rmse(project: Path) -> float:
    """Pull the normalized reprojection error from OpenSfM's stats.json."""
    stats = project / "opensfm" / "stats" / "stats.json"
    data = json.loads(stats.read_text())
    rec = data["reconstruction_statistics"]
    # Prefer the normalized error; fall back to the raw pixel RMSE.
    return float(rec.get("reprojection_error_normalized",
                         rec.get("reprojection_error", 99.0)))

def run_with_constraints(datasets: Path, project: str, gps_accuracy_m: float,
                         lens: str = "brown", timeout_s: int = 86_400) -> None:
    """Re-run ODM with priors that match capture reality and a fixed lens model."""
    cmd = [
        "docker", "run", "--rm", "-v", f"{datasets}:/datasets",
        "opendronemap/odm:3.5.4", "--project-path", "/datasets", project,
        "--gps-accuracy", str(gps_accuracy_m),   # standalone ~5.0, RTK ~0.05
        "--camera-lens", lens,                    # brown / perspective / fisheye
        "--dsm", "--orthophoto-resolution", "2.0",
    ]
    gcp = datasets / project / "gcp_list.txt"
    if gcp.is_file():
        cmd += ["--force-gps"]                     # let GCPs anchor over weak GPS
        logging.info("GCP file present; anchoring datum on surveyed control.")
    proc = subprocess.run(cmd, capture_output=True, text=True, timeout=timeout_s)
    if proc.returncode != 0:
        raise RuntimeError(f"ODM exit {proc.returncode}:\n{(proc.stderr or '')[-1500:]}")

def repair_divergence(datasets: Path, project: str, threshold: float = 1.0) -> None:
    """If the reconstruction diverged, loosen priors and re-run once."""
    rmse = read_reprojection_rmse(datasets / project)
    logging.info("reprojection RMSE = %.3f (threshold %.2f)", rmse, threshold)
    if rmse > threshold:
        # A high RMSE with standalone GPS almost always means over-tight priors.
        run_with_constraints(datasets, project, gps_accuracy_m=5.0, lens="brown")

The --gps-accuracy 5.0 value is deliberate: it tells the solver the camera positions are trustworthy to about five metres, so it stops distorting the geometry to hit centimetre-wrong GNSS fixes and lets the tie-point network define the shape. If you actually flew RTK/PPK, tighten it toward 0.05 instead — the failure mode there is the opposite, a loose prior that lets the block drift. Setting --camera-lens explicitly (rather than letting ODM auto-detect on a sparse block) stops the solver from absorbing distortion into pose. When a gcp_list.txt is present, anchoring on surveyed control is the durable fix; producing that file correctly is covered in automating GCP detection with Python.

Fix order for a diverging reconstruction Four remedies arranged left to right in the order they should be attempted, with a note on cost and reversibility under each. First, enable a robust loss, which is a one-line configuration change and almost always the right first move. Second, tighten the outlier threshold used during matching, which removes the offending observations before the solver sees them. Third, supply better initial poses from validated GPS, which shrinks the basin the optimiser has to search. Fourth, and only if the first three fail, reduce the problem: split the block so a local defect cannot propagate. An arrow beneath notes that each step costs more than the one before it. 1 · robust loss soft_l1 or huber f_scale ≈ 2 px one line, reversible fixes most cases 2 · tighten matching lower ratio threshold stricter RANSAC band re-run matching minutes, not hours 3 · better priors validated EXIF or PPK poses seeded, not guessed re-run ingestion shrinks the search basin 4 · split the block contain the defect merge afterwards full re-run last resort, adds seams increasing cost and decreasing reversibility → Work left to right and stop at the first step that converges. Jumping straight to step four hides the cause and leaves it to recur on the next flight.

Figure 2 — Divergence has four standard remedies and they are not interchangeable. The cheap ones also happen to be the ones that identify the cause, so trying them in order is both faster and more informative.

Edge-case matrix

The signature in the log and stats tells you which constraint to correct. Applying the wrong fix — tightening priors on a drifting RTK block, for instance — makes divergence worse.

Input variant Downstream symptom if unchecked Expected handling
Standalone GPS, tight prior Edge-bowing / doming, RMSE climbs Loosen --gps-accuracy to ~5.0 m
RTK/PPK, loose prior Whole block drifts off true position Tighten --gps-accuracy to ~0.05 m
Thin cross-strip overlap Under-constrained cameras drift, oscillating RMSE Add overlap or GCPs; treat sparse edges as missing
Fisheye lens, perspective model Residuals never converge, warped intrinsics Set --camera-lens fisheye to match the sensor
One mistyped GCP coordinate Model sheared toward the bad control point Remove/fix the GCP; check its CRS and sign
GCP in wrong CRS Systematic offset plus divergence Reproject control to the project CRS first
Convergence signature before and after the robust loss Two cost-versus-iteration traces on the same axes for the same dataset. The first, under a squared loss, falls for three iterations and then climbs steadily away, ending an order of magnitude above where it started. The second, with a soft-L1 loss and a two-pixel scale, falls smoothly and flattens by iteration twelve at a cost roughly two orders of magnitude below the starting value. A marker at iteration three shows where the two traces separate, which is the moment the outlier begins to dominate the squared cost. 0 5 10 15 20 10⁰ 10¹ 10² 10³ 10⁴ solver iteration total cost squared loss — diverging soft_l1 — converged traces separate at iteration 3

Figure 3 — The two runs are identical up to the third iteration. That is the point at which the outlier’s squared contribution starts to exceed everything else, and it is why “it ran fine for a few iterations” is not evidence that the data is sound.

Verification snippet

Confirm the re-run actually converged rather than just finishing: assert the reprojection RMSE dropped below threshold and that the observation count did not collapse (a “converged” solve on almost no points is not a fix).

from pathlib import Path

def verify_convergence(project: Path, threshold: float = 1.0,
                       min_observations: int = 5000) -> None:
    """Assert the bundle adjustment converged on a healthy observation set."""
    import json
    rec = json.loads((project / "opensfm" / "stats" / "stats.json").read_text())
    stats = rec["reconstruction_statistics"]
    rmse = float(stats.get("reprojection_error_normalized",
                           stats.get("reprojection_error", 99.0)))
    observations = int(stats.get("observations_count", 0))

    assert rmse <= threshold, f"still diverged: RMSE {rmse:.3f} > {threshold}"
    assert observations >= min_observations, \
        f"converged on too few observations ({observations}); geometry is thin"
    print(f"converged: RMSE {rmse:.3f} over {observations} observations")

A pass here means the solver settled on a low, stable residual over a dense tie-point network — the two conditions that together certify a metrically sound block. Track the RMSE across runs so a regression is caught immediately rather than at delivery.

When to escalate

Escalate when the constraint is correct but the geometry itself cannot support a solve:

  • RMSE stays high even with matched priors and a correct lens. The tie-point network is too thin. This is a coverage problem — fix it upstream with the flight overlap validation routine, which rejects sparse blocks before reconstruction.
  • You need to tune the solver’s loss function or Jacobian directly. ODM’s flags have a ceiling; the robust-loss and residual-weighting internals live in optimizing bundle adjustment with Python.
  • The divergence is really a GCP residual problem. If control points are fighting each other, the diagnosis belongs with GCP error distribution rather than the solver — return through setting up OpenDroneMap with Python and audit the control network.

← Setting Up OpenDroneMap with Python