Tuning the SIFT Contrast Threshold for Low-Texture Scenes

Half the block reconstructs beautifully and the half over the sand quarry does not. The detector is returning a few hundred keypoints per frame there against four thousand elsewhere, matching collapses, and the reconstruction either drops those frames or splits. The scene has real structure — a person can see the ripples in the sand — and the detector is rejecting it because the contrast between adjacent pixels is below a threshold tuned for ordinary photographs.

This page sets that threshold from the imagery rather than by bisection, and identifies the case where lowering it further only adds noise.

What the threshold actually rejects

SIFT locates candidate keypoints as extrema in a difference-of-Gaussian pyramid, then discards any whose response magnitude falls below contrastThreshold. The default, 0.04, was chosen against natural photographs with strong local contrast. Aerial imagery of uniform ground routinely has a difference-of-Gaussian response an order of magnitude smaller, so the filter that removes noise on a street scene removes signal on a sand flat.

Two properties make this tractable. The response distribution is smooth and unimodal, so a percentile of the observed responses is a principled threshold rather than a guess. And the noise floor is measurable: a frame’s sensor noise sets a level below which detections cannot be real, and it can be estimated from a flat region of the same frame.

Between those two numbers there is usually a wide usable band, and outside it there is nothing to recover.

Difference-of-Gaussian response distributions on three surfaces Three response distributions plotted on a common axis of difference-of-Gaussian magnitude. Over an urban scene the distribution extends well past the default threshold of zero point zero four, so the default keeps thousands of keypoints. Over pasture the distribution sits mostly below the default, keeping a few hundred. Over dry sand almost the entire distribution lies below the default and the detector returns almost nothing. A vertical line marks the default threshold and a shaded band marks the sensor noise floor, showing that the sand distribution still sits above the noise floor and is therefore recoverable by lowering the threshold. noise default 0.04 difference-of-Gaussian response magnitude → keypoints urban pasture dry sand The sand distribution sits above the noise floor and below the default — which is exactly the recoverable case.

Figure 1 — Where the default lands on three surfaces. The threshold is not wrong in general; it is wrong relative to the response distribution of this particular imagery.

Minimal reproducible solution

Set the threshold from a target keypoint count on a sample of frames, bounded below by an estimate of the noise floor.

import cv2
import numpy as np


def noise_floor(gray: np.ndarray) -> float:
    """Estimate the DoG response attributable to sensor noise alone.

    Uses the median absolute deviation of a Laplacian, which is dominated by
    noise on smooth imagery and robust to the few real edges present.
    """
    lap = cv2.Laplacian(gray, cv2.CV_32F, ksize=3)
    mad = float(np.median(np.abs(lap - np.median(lap))))
    return 1.4826 * mad / 255.0          # to the 0–1 scale SIFT uses


def calibrate_contrast_threshold(frames: list[np.ndarray],
                                 target_keypoints: int = 3000,
                                 lo: float = 0.002, hi: float = 0.04,
                                 iterations: int = 8) -> float:
    """Largest threshold that still yields the target count on the median frame.

    Bisection on the threshold rather than on the count, because the count is
    monotone in the threshold and the relationship is otherwise unknown.
    """
    floor = max(noise_floor(f) for f in frames)
    lo = max(lo, floor)
    if lo >= hi:
        raise ValueError(
            f"noise floor {floor:.4f} is at or above the default threshold — "
            "this imagery has no recoverable texture; see the escalation notes")

    for _ in range(iterations):
        mid = (lo + hi) / 2.0
        det = cv2.SIFT_create(contrastThreshold=mid, edgeThreshold=10)
        counts = [len(det.detect(f, None)) for f in frames]
        if int(np.median(counts)) >= target_keypoints:
            lo = mid                      # can afford to be stricter
        else:
            hi = mid
    return lo

Two aspects matter. Bisecting downward from the default rather than upward from zero returns the largest threshold that meets the target, which keeps as much of the noise rejection as the imagery allows. And clamping at the measured noise floor is what makes the routine safe: without it, a genuinely featureless frame drives the threshold to zero and the detector returns tens of thousands of pure-noise keypoints, which is far worse than returning none.

Calibrating on a sample rather than per frame is deliberate too. A per-frame threshold makes descriptors from different frames incomparable in a subtle way — the same physical feature can be kept in one frame and rejected in its neighbour — so the threshold should be constant across a block, or at least across a surface class within it.

Edge-case matrix

Surface Typical yield at 0.04 Calibrated threshold Outcome
Urban, mixed 4 000+ 0.04 (unchanged) No action needed
Pasture, crops 800–2 000 0.015–0.025 Recovers to target
Dry sand, gravel 100–400 0.006–0.012 Recovers, more outliers
Fresh snow < 100 at the noise floor Marginal; expect a weak block
Still water ~0 below the noise floor Unrecoverable — mask it
Uniform roof, solar array 200–600 0.008–0.015 Recovers, but repetitive
Deep shadow < 50 locally unchanged globally Exposure problem, not threshold
Motion-blurred frame Halves unchanged Blur, not contrast; reject the frame

The last three rows are the ones worth internalising. A repetitive surface recovers plenty of keypoints and matches them wrongly, which is a ratio-test problem rather than a detection one. A shadowed region and a blurred frame are not low-contrast in the sense the threshold addresses — lowering it globally to chase them degrades every other frame.

Verification snippet

Keypoint count is the wrong success criterion on its own: the threshold can always be lowered until the count is met. The measure that matters is how many of those keypoints survive geometric verification.

def inlier_yield(detector, matcher, pairs) -> float:
    """Fraction of detected keypoints that end up as verified inliers."""
    detected = inliers = 0
    for left, right in pairs:
        kp_l, des_l = detector.detectAndCompute(left, None)
        kp_r, des_r = detector.detectAndCompute(right, None)
        detected += len(kp_l) + len(kp_r)
        if des_l is None or des_r is None or len(kp_l) < 8 or len(kp_r) < 8:
            continue
        good = [m for m, n in matcher.knnMatch(des_l, des_r, k=2)
                if m.distance < 0.75 * n.distance]
        if len(good) < 8:
            continue
        import numpy as np
        src = np.float32([kp_l[m.queryIdx].pt for m in good]).reshape(-1, 1, 2)
        dst = np.float32([kp_r[m.trainIdx].pt for m in good]).reshape(-1, 1, 2)
        _, mask = cv2.findFundamentalMat(src, dst, cv2.FM_RANSAC, 1.0, 0.999)
        inliers += int(mask.sum()) if mask is not None else 0
    return inliers / max(detected, 1)


def assert_threshold_helps(before: float, after: float,
                           min_gain: float = 1.5) -> None:
    """Lowering the threshold must raise the inlier yield, not just the count."""
    assert after >= before * min_gain, (
        f"inlier yield went from {before:.3f} to {after:.3f} — the extra "
        "keypoints are not matching, so the threshold is now below the "
        "useful signal")

That assertion is the guard against the failure this whole exercise invites: a threshold low enough to produce four thousand keypoints per frame and an inlier yield lower than before, which means the additional detections are noise and the matching stage is now doing more work for a worse result.

Keypoint count and inlier yield as the threshold falls Two curves plotted against a falling contrast threshold. Keypoint count rises monotonically as the threshold falls, without limit. Inlier yield — the fraction of detected keypoints that survive geometric verification — rises to a peak and then falls away as detections below the useful signal begin to dominate. A shaded band marks the region between the peak of the yield curve and the point where it has fallen back to its starting value, labelled as the usable range. A note observes that optimising on count alone always chooses a threshold to the right of this band. usable range 0.04 0.02 0.01 0.005 0.002 contrast threshold, falling → keypoint count inlier yield peaks here Optimising on count alone lands to the right of the band, where every extra keypoint costs matching time and returns nothing.

Figure 2 — Why the target is a yield rather than a count. The two curves diverge precisely where the threshold stops recovering signal and starts admitting noise.

When to escalate

  • The noise floor is at or above the default threshold. There is no threshold that separates signal from noise, because there is no signal. Still water and fresh snow behave this way; the fix is to exclude those regions before detection, as in masking sky and water before feature detection.
  • Count recovers, yield does not, and the surface is repetitive. Detection was never the constraint. Crop rows and roof tiles produce plenty of distinctive-looking keypoints that match the wrong instance, which is diagnosed in fixing “not enough inliers” RANSAC failures.
  • Yield improves and the reconstruction still splits there. The frames now match each other and not the rest of the block, which is a connectivity problem rather than a detection one; see fixing an OpenSfM reconstruction that split into multiple submodels.

← Feature Detection Algorithms for Drone Imagery

One threshold per surface class, not one per frame A survey block containing three surface classes: an urban strip, pasture, and a sand quarry. Three approaches are compared. A single global threshold serves the urban strip and starves the quarry. A per-frame threshold makes the same physical feature detectable in one frame and rejected in its neighbour, because adjacent frames straddling a boundary calibrate differently. A per-class threshold, assigned from a coarse surface classification, keeps the threshold constant wherever frames overlap and is the arrangement that works. urban pasture sand quarry one global threshold serves the left, starves the right one per frame neighbouring frames disagree about the same feature one per surface class constant wherever frames overlap — the arrangement that works Matching compares descriptors across frames, so the threshold has to be constant across any pair that will be matched.

Figure 3 — The granularity question. Per-frame calibration looks more adaptive and quietly breaks the assumption that two overlapping frames were detected the same way.