Merging Submodels with Shared Control Points

The reconstruction finished and produced three models instead of one. Each covers part of the site, each is internally consistent, and none of them is georeferenced the same way as the others — two are in arbitrary frames and one picked up the control points.

Merging them is a registration problem: find the similarity transform — rotation, translation and scale — that brings each submodel into a common frame, then re-run the adjustment over the combined set. This page covers estimating that transform robustly, choosing what to merge on, and recognising the cases where a merge should be refused rather than forced. It follows the diagnosis in matching strategies for large and linear surveys.

# What can be merged on

Three kinds of correspondence can tie two submodels together, and they differ in strength.

Shared control points are the strongest. A control point visible in both submodels has a known real-world coordinate, so each submodel can be transformed into the real frame independently and the merge is not really a registration at all. Three well-distributed shared points are sufficient; four or more allow a residual check.

Shared camera positions work when both submodels contain the same image, which happens when a split occurred mid-line. The camera centres are estimated rather than measured, so the transform inherits their error, but there are usually many of them.

Shared tie points are weakest and most plentiful. They require the matching to have found correspondences across the split, which is precisely what failed — but a targeted re-match limited to the boundary region often succeeds where the global pass did not.

Figure 1 — Three routes to a merge, in decreasing order of strength.

# Minimal reproducible solution

import numpy as np


def similarity_transform(source: np.ndarray, target: np.ndarray) -> dict:
    """Least-squares rotation, scale and translation mapping source onto target.

    The closed-form solution — Umeyama's — is exact for the least-squares
    problem and needs no iteration, which matters because it will be called
    inside a RANSAC loop.
    """
    src = np.asarray(source, dtype=float)
    tgt = np.asarray(target, dtype=float)
    if src.shape != tgt.shape or src.shape[0] < 3:
        raise ValueError("need at least three matched points of the same shape")

    mu_s, mu_t = src.mean(axis=0), tgt.mean(axis=0)
    cs, ct = src - mu_s, tgt - mu_t
    covariance = ct.T @ cs / len(src)

    U, D, Vt = np.linalg.svd(covariance)
    S = np.eye(3)
    if np.linalg.det(U) * np.linalg.det(Vt) < 0:
        S[2, 2] = -1.0                       # forbid a reflection

    R = U @ S @ Vt
    var_s = float((cs ** 2).sum() / len(src))
    scale = float(np.trace(np.diag(D) @ S) / max(var_s, 1e-12))
    t = mu_t - scale * R @ mu_s

    residuals = tgt - (scale * (R @ src.T).T + t)
    return {"R": R, "scale": scale, "t": t,
            "rms_m": float(np.sqrt((residuals ** 2).sum(axis=1).mean())),
            "max_m": float(np.linalg.norm(residuals, axis=1).max()),
            "points": len(src)}

Forbidding a reflection is the detail that prevents a spectacular failure. Without the determinant correction, a poorly conditioned point set can produce a mirrored transform that fits the points and turns the submodel inside out — visually obvious and arithmetically plausible.

# Rejecting bad correspondences

A single mismatched point drags a least-squares similarity transform substantially, so the estimation belongs inside a robust loop.

import numpy as np


def robust_similarity(source: np.ndarray, target: np.ndarray,
                      *, threshold_m: float = 0.5, iterations: int = 500,
                      seed: int = 0) -> dict:
    """RANSAC over similarity transforms, with a final fit on the inliers."""
    rng = np.random.default_rng(seed)
    n = len(source)
    if n < 4:
        return {**similarity_transform(source, target), "inliers": list(range(n)),
                "note": "too few points for a robust fit"}

    best_inliers: list[int] = []
    for _ in range(iterations):
        sample = rng.choice(n, size=3, replace=False)
        try:
            fit = similarity_transform(source[sample], target[sample])
        except (ValueError, np.linalg.LinAlgError):
            continue
        predicted = fit["scale"] * (fit["R"] @ source.T).T + fit["t"]
        errors = np.linalg.norm(target - predicted, axis=1)
        inliers = np.flatnonzero(errors < threshold_m).tolist()
        if len(inliers) > len(best_inliers):
            best_inliers = inliers

    if len(best_inliers) < 3:
        return {"merged": False, "note": "no consistent transform found"}
    final = similarity_transform(source[best_inliers], target[best_inliers])
    return {**final, "inliers": best_inliers,
            "inlier_fraction": len(best_inliers) / n, "merged": True}

Figure 3 — Four stages, and the last is what makes it a model rather than a mosaic.

# Edge-case matrix

The collinear case is the one that catches corridor surveys. Three control points along a road are collinear, and a similarity transform fitted to them is free to rotate about that line — so the merged model is correct along the corridor and can be tilted across it. Any point off the line resolves it.

import numpy as np


def check_geometry(points: np.ndarray, *, min_spread_ratio: float = 0.05) -> dict:
    """Is this point set well conditioned for a similarity transform?"""
    centred = points - points.mean(axis=0)
    singular = np.linalg.svd(centred, compute_uv=False)
    ratio = float(singular[-1] / max(singular[0], 1e-12))
    return {"singular_values": singular.tolist(),
            "conditioning_ratio": ratio,
            "well_conditioned": ratio > min_spread_ratio,
            "note": ("points span three dimensions adequately" if ratio > min_spread_ratio
                     else "points are nearly collinear or coplanar — the transform is "
                          "free to rotate about the degenerate direction")}

# Verification snippet

import numpy as np


def verify_merge(transform: dict, holdout_source: np.ndarray,
                 holdout_target: np.ndarray, *, tolerance_m: float = 0.1) -> dict:
    """Apply the transform to points it was not fitted on.

    A merge that fits its own correspondences and fails on held-out ones has
    fitted the errors, which is exactly what a clustered or minimal
    correspondence set produces.
    """
    predicted = transform["scale"] * (transform["R"] @ holdout_source.T).T + transform["t"]
    errors = np.linalg.norm(holdout_target - predicted, axis=1)
    return {"holdout_points": len(errors),
            "rms_m": float(np.sqrt((errors ** 2).mean())),
            "max_m": float(errors.max()),
            "acceptable": float(np.sqrt((errors ** 2).mean())) < tolerance_m,
            "ratio_to_fit": float(np.sqrt((errors ** 2).mean())
                                  / max(transform.get("rms_m", 1e-9), 1e-9))}

A holdout RMS more than about twice the fit RMS means the correspondences were not representative — usually because they were clustered in one part of the overlap. The remedy is more correspondences spread across the shared area, not a better solver.

Figure 2 — Why correspondence distribution matters more than correspondence count.

# When to refuse the merge

Not every split should be merged. Three situations argue for delivering the submodels separately or re-processing instead.

No genuine overlap. If the submodels cover disjoint areas, there is nothing to register on and any transform is an extrapolation. Deliver them as separate products with their own georeferencing.

Overlap without structure. Water, bare soil or a featureless field between two blocks provides correspondences that are unreliable, and a merge built on them will be locally plausible and globally wrong.

A holdout that will not come down. If the merge cannot achieve the survey’s tolerance across the shared area, forcing it produces a product that is worse than either submodel. Re-processing with a better pair graph is the right answer.

After a successful merge, the combined model should be re-adjusted rather than left as two transformed pieces. The similarity transform is a rigid placement; a joint bundle adjustment over the combined observations distributes the residual error properly and is what makes the merged model as good as one that never split.

# When to escalate

• Submodels keep appearing on every run. The pair graph is the problem, not the merge. Fix the matching rather than merging repeatedly.
• The merge succeeds and the combined model has a discontinuity. The transform was applied without a joint re-adjustment. Re-run the bundle over the merged observations.
• Control exists in only one submodel. Merge first, then georeference the combined model. Transforming an ungeoreferenced submodel onto a georeferenced one is exactly what the similarity transform is for.

# Related

• Matching Strategies for Large and Linear Surveys
• Fixing an OpenSfM Reconstruction Split Into Multiple Submodels
• Distributing GCP Errors Across Orthomosaics

← Matching Strategies for Large and Linear Surveys