Fixing an OpenSfM Reconstruction Split Into Multiple Submodels

reconstruction.json contains two entries where you expected one. Each is internally excellent — sub-pixel residuals, clean camera track — and the two are unrelated in scale, rotation and position. The orthomosaic, if it renders at all, shows one half correct and the other rotated across it.

The cause is always the same: the match graph is disconnected. The remedy is not always the same, because sometimes the connection is recoverable from imagery you already have and sometimes it was never flown.

Why the reconstruction splits rather than degrading

Incremental structure from motion grows a reconstruction from a seed pair by repeatedly adding the image with the most correspondences to what is already reconstructed. When no unreconstructed image has enough correspondences with the current model, the process stops and starts a new model from the best remaining pair. Two models therefore mean the verified pair graph has two components — not that the imagery is poor, but that no surviving pair of images spans the gap.

The distinction between “few correspondences” and “no verified pairs” matters. A weak bridge with 30 inliers keeps the block together and produces a badly conditioned join. A bridge with 8 inliers is dropped at verification and produces a clean split. The second is more visible and, oddly, easier to fix.

Three conditions cause it, and they are separable from the graph alone.

A coverage gap. The aircraft stopped shooting — a battery swap, a data card change, an operator pause — and there is genuinely no imagery over a strip of ground.

A featureless corridor. Imagery exists across the gap and it is over water, sand or fresh snow, so the pairs spanning it produce too few inliers to survive verification.

A pair-selection gap. The imagery and the features exist, and the candidate pairs that would have connected the halves were never attempted, because the selection radius was too small or the positions used to select them were wrong.

Three causes of a disconnected match graph Three flight footprints, each producing two reconstruction components for a different reason. In the first, a coverage gap, no images exist over a strip of ground, so no pair can span it. In the second, a featureless corridor, images exist across the gap but lie over water so the spanning pairs yield too few inliers to survive verification. In the third, a pair-selection gap, both imagery and features exist but the candidate pairs across the gap were never attempted because the spatial selection radius was too small. Each is annotated with the evidence that identifies it and whether it is recoverable without re-flying. coverage gap no images evidence: a hole in the position coverage re-fly required featureless corridor water evidence: pairs attempted, inliers below the floor sometimes recoverable pair-selection gap never attempted evidence: no candidate pairs across the gap fully recoverable The three are told apart by whether the pairs exist, were attempted, and survived — three counts, not a judgement.

Figure 1 — Three causes with three different prognoses. Only the first requires flying again, and it is the one most often assumed when either of the others is the real condition.

Minimal reproducible solution

Read the components from the reconstruction, then ask the pair graph which of the three conditions produced them.

import json
from collections import defaultdict
from pathlib import Path


def components(reconstruction_json: Path) -> list[set[str]]:
    """Image sets per reconstructed model, largest first."""
    models = json.loads(reconstruction_json.read_text())
    return sorted((set(m["shots"].keys()) for m in models), key=len, reverse=True)


def diagnose_split(project: Path) -> str:
    """Name the condition from the candidate and verified pair counts."""
    comps = components(project / "opensfm" / "reconstruction.json")
    if len(comps) < 2:
        return "single component — nothing to diagnose"
    a, b = comps[0], comps[1]

    candidates = _load_candidate_pairs(project)     # what was attempted
    verified = _load_verified_pairs(project)        # what survived RANSAC

    across_candidates = [(i, j) for i, j in candidates
                         if (i in a and j in b) or (i in b and j in a)]
    across_verified = [(i, j) for i, j in verified
                       if (i in a and j in b) or (i in b and j in a)]

    if not across_candidates:
        return ("pair-selection gap: no cross-component pairs were attempted. "
                "Widen the selection radius or fix the positions it uses.")
    if not across_verified:
        return (f"{len(across_candidates)} cross-component pairs attempted, "
                "none verified: the overlap region has no matchable texture, "
                "or the verification threshold is too strict.")
    return (f"{len(across_verified)} cross-component pairs verified but the "
            "reconstruction still split — the bridge is below the minimum "
            "track count the incremental step requires.")

Distinguishing “not attempted” from “attempted and rejected” is the entire diagnostic. The first is a configuration problem with a free fix; the second is a data problem whose fix costs something; and the third — verified pairs that exist and are too few — is a threshold that can be relaxed deliberately, accepting a weaker join in exchange for one block.

The fix for the recoverable cases is to force the missing pairs rather than to widen the selection globally, which would multiply matching work across the whole survey.

def force_bridge_pairs(project: Path, a: set[str], b: set[str],
                       positions: dict, k: int = 12) -> list[tuple[str, str]]:
    """The k closest cross-component pairs, appended to the candidate list.

    Targets the join specifically instead of raising the global radius, which
    would add work on every pair in the survey to fix one boundary.
    """
    import numpy as np
    pairs = []
    for i in a:
        for j in b:
            d = np.linalg.norm(np.array(positions[i]) - np.array(positions[j]))
            pairs.append((d, i, j))
    pairs.sort()
    chosen = [(i, j) for _, i, j in pairs[:k]]

    path = project / "opensfm" / "candidate_pairs_extra.txt"
    path.write_text("\n".join(f"{i} {j}" for i, j in chosen))
    return chosen

Edge-case matrix

Situation Diagnostic output Action
Battery swap mid-block no cross candidates, positions show a hole Re-fly the strip
Two flights, one project no cross candidates, two clusters Correct; merge on control instead
Water between two shores candidates attempted, none verified Re-fly a crossing, or merge on control
Selection radius too small no cross candidates, positions contiguous Force the bridge pairs
Wrong EXIF positions no cross candidates, positions implausible Fix positions; selection was blind
Bridge verified but thin few verified pairs Relax the track minimum, accept a weak join
Three or more components apply pairwise Fix the largest join first; it often fixes the rest
Components of very unequal size small one is a few frames Often simplest to drop the fragment

Two rows deserve emphasis. Two flights in one project is not a failure at all — the correct handling is to reconstruct them separately and co-register on ground control, exactly as for mixed sensor payloads. And wrong EXIF positions produce a selection gap that looks like a coverage gap, because selection is spatial: positions that are wrong send the radius search to the wrong neighbours.

Verification snippet

After forcing the bridge, verify connectivity on the graph before spending hours on a reconstruction.

def assert_graph_connected(verified_pairs, images) -> None:
    """One connected component over the verified pair graph, with margin."""
    parent = {im: im for im in images}

    def find(x):
        while parent[x] != x:
            parent[x] = parent[parent[x]]
            x = parent[x]
        return x

    for i, j in verified_pairs:
        ri, rj = find(i), find(j)
        if ri != rj:
            parent[ri] = rj

    roots = {find(im) for im in images}
    assert len(roots) == 1, (
        f"verified pair graph still has {len(roots)} components — "
        "reconstruction will split again")

    # A single bridging pair is a connection that one bad match can sever.
    degree = {im: 0 for im in images}
    for i, j in verified_pairs:
        degree[i] += 1
        degree[j] += 1
    weak = [im for im, d in degree.items() if d < 2]
    assert not weak, f"{len(weak)} image(s) hang off a single pair: {weak[:5]}"

The second assertion is the one worth keeping permanently. A graph that is connected through exactly one pair is connected in the same sense that a rope bridge is a road: it survives until one match is rejected on a re-run, at which point the block splits again and the cause looks intermittent.

A single bridge against a redundant one Two match graphs joining the same pair of image clusters. In the first, one verified pair connects the two clusters, so the graph is technically connected and losing that single pair on a subsequent run splits the block again. In the second, four verified pairs span the gap, so the connection survives the loss of any one of them and the join is also better conditioned because the relative geometry is over-determined. A note observes that a connectivity check alone passes both, which is why the minimum-degree check matters. one bridging pair lose this pair and the block splits again four bridging pairs survives losing any one, and the join is over-determined A connectivity check passes both graphs. Only a minimum-degree check separates them. Which is why an "intermittent" split that appears on some runs and not others is almost always the left-hand graph.

Figure 2 — Connected is not the same as robustly connected. The left graph passes every connectivity test and fails on the next run when one match falls below threshold.

When to escalate

  • The gap is genuinely unflown. No amount of processing recovers imagery that does not exist. Reconstruct the components separately and co-register them on shared ground control, which is a defensible product; a forced merge without correspondences is not.
  • Forcing the bridge succeeds and the join is visibly distorted. The bridging pairs exist and are weak, so the relative orientation between the halves is poorly determined. Adding control points near the join constrains it directly and is more reliable than adding more marginal pairs.
  • The split recurs on some runs and not others. The bridge is a single pair sitting near the verification threshold. Force additional pairs across the join rather than lowering the threshold globally, which would admit weak pairs everywhere.

← Troubleshooting Alignment and Matching Failures

Cost of the three remedies Three remedies for a split reconstruction, ordered by cost. Forcing a dozen targeted cross-component pairs costs a few minutes of matching and no flying. Widening the global selection radius costs a full re-match of the survey, which is hours, to fix one boundary. Re-flying the gap costs a field visit. A note recommends attempting them strictly in that order, since the cheapest remedy also produces the clearest evidence about whether the next one is necessary. force 12 targeted pairs matching only the join minutes try first, always widen the global radius re-match the whole survey hours fixes one boundary, costs everywhere re-fly the gap a field visit days the only fix for a real coverage hole Work left to right, because the cheapest attempt also produces the evidence that justifies the next one. Twelve pairs that fail to verify are proof the overlap has no matchable texture — which no amount of re-matching will change.

Figure 3 — Ordering the remedies. The targeted attempt is cheap enough to run before the diagnosis is certain, and its failure is itself the diagnosis.