Feature Detection Algorithms for Drone Imagery

Feature detection is the first compute-heavy stage of any photogrammetric reconstruction, and the detector you choose decides how the rest of the pipeline behaves. This page walks through selecting, configuring, and batch-running keypoint detectors on UAV-captured imagery in Python — the exact scenario a surveying technician or GIS developer faces when a multi-hundred-image survey block has to be turned into a dense, well-distributed tie-point network without exhausting workstation RAM. Everything here feeds the broader automated image alignment and feature matching workflows, where these descriptors are matched, filtered with RANSAC, and handed to bundle adjustment.

Audience prerequisites: Python 3.10+, working knowledge of numpy array dtypes, and a machine with at least 16 GB RAM (32 GB recommended for SIFT on full-resolution frames). You should already have your imagery organised on disk; if not, set up the batch processing directory structure first, because the extractor below assumes one flight strip per directory.

# Prerequisites

Pin these in a requirements.txt and install into a fresh virtual environment. The opencv-contrib-python and base opencv-python packages must not both be installed — the duplicate cv2 namespaces shadow each other and SIFT silently disappears.

# Conceptual Architecture

Detection sits between image ingestion and pairwise matching. Each frame is read as a single-band 2D array, normalised to 8-bit, and passed to a detector that returns keypoints (pixel coordinates, scale, orientation) plus a descriptor matrix. Crucially, the GeoTIFF’s affine transform and CRS travel with the descriptors so that later stages can map pixel coordinates back to ground coordinates without re-opening the rasters. Descriptors are persisted out-of-core immediately, so a survey block never has to sit in memory all at once — the same determinism principle the parent pipeline depends on.

UAV imagery stresses detectors in specific ways: high forward/side overlap produces near-duplicate frames, repetitive textures (agricultural fields, industrial rooftops, water) generate ambiguous matches, and illumination drifts across long flight lines. The detector and its parameters must be chosen against those conditions rather than left at library defaults.

# 1. Choosing a Detector

Three detectors cover almost every UAV mapping case:

• SIFT (Scale-Invariant Feature Transform) — the accuracy baseline. Strong scale and rotation invariance, dense well-distributed keypoints, but 128-dimension float32 descriptors are memory-hungry and frequently trigger out-of-memory (OOM) failures on full-resolution blocks.
• ORB (Oriented FAST and Rotated BRIEF) — fast, with compact 32-byte binary descriptors that scale to consumer hardware. Weaker under large viewpoint change and low-contrast surfaces.
• AKAZE — a middle ground that preserves edge fidelity at reduced descriptor dimensionality, useful for infrastructure inspection where structural edges dominate.

The choice is not fixed at design time. Drive it from measured inlier ratios, flight ground sample distance (GSD), and available RAM. A head-to-head benchmark with threshold calibration lives in fixing SIFT vs ORB performance in UAV photos.

import cv2

def build_detector(detector_type: str = "SIFT", max_features: int = 8000):
    """Return a configured OpenCV detector. ORB/AKAZE fall back gracefully
    when the contrib SIFT module is unavailable."""
    detector_type = detector_type.upper()
    if detector_type == "SIFT":
        # contrastThreshold/edgeThreshold tuned for aerial textures below.
        return cv2.SIFT_create(
            nfeatures=max_features,
            contrastThreshold=0.04,
            edgeThreshold=10,
        )
    if detector_type == "AKAZE":
        return cv2.AKAZE_create(threshold=0.001)
    return cv2.ORB_create(nfeatures=max_features, scaleFactor=1.2, nlevels=8)

Figure 2 — The trade the detector choice actually buys. SIFT’s advantage is invariance; its cost is a descriptor sixteen times the size of ORB’s, which is what turns a full-resolution block into an out-of-memory failure.

# Why Aerial Frames Defeat the Default Settings

Every detector in OpenCV ships with thresholds tuned on ground-level photographs: close subjects, strong perspective, high local contrast, and a scene that fills the frame at wildly varying depths. A nadir UAV frame is almost the opposite. The scene is effectively planar, the scale is uniform across the image, the illumination is flat, and large regions carry no structure at all. Three consequences follow, and each one has a different fix.

The first is contrast starvation. SIFT’s contrastThreshold rejects candidate keypoints whose difference-of-Gaussian response is weak, and on a hazy 120 m flight over pasture a large fraction of genuinely useful features fall below the default 0.04. The visible symptom is a keypoint count that collapses on some frames and not others within the same flight — usually the frames shot into the sun or through thin cloud. Lowering the threshold to 0.02–0.03 recovers them at the cost of more spurious detections, which the geometric verification stage then has to discard.

The second is spatial clumping. Detectors return the strongest responses, and on a survey frame the strongest responses cluster wherever the scene happens to be busy: a treeline, a car park, a building edge. Two thousand keypoints all sitting in one corner constrain the camera pose far worse than four hundred spread across the frame, because the geometry is nearly degenerate. Capping nfeatures does not help — it keeps the strongest, which are the clustered ones. What helps is retaining keypoints per grid cell, so every part of the frame contributes.

The third is repeated texture. Crop rows, roof tiles, solar arrays and railway sleepers produce descriptors that are genuinely similar because the world genuinely repeats. These are not detection failures; they are matching failures waiting to happen, and they are the reason the ratio test in geometric verification matters more on aerial data than on ordinary photographs.

import numpy as np


def grid_filter(keypoints, descriptors, shape, cells=(6, 6), per_cell=60):
    """Keep the strongest keypoints per grid cell so coverage stays uniform.

    Detectors return globally strongest responses, which on a survey frame
    means a cluster over the one busy region. Uniform coverage constrains the
    pose far better than a larger, clumped set.
    """
    h, w = shape
    ch, cw = h / cells[0], w / cells[1]
    keep: list[int] = []
    buckets: dict[tuple[int, int], list[int]] = {}
    for i, kp in enumerate(keypoints):
        cell = (min(int(kp.pt[1] / ch), cells[0] - 1),
                min(int(kp.pt[0] / cw), cells[1] - 1))
        buckets.setdefault(cell, []).append(i)
    for idx in buckets.values():
        idx.sort(key=lambda i: keypoints[i].response, reverse=True)
        keep.extend(idx[:per_cell])
    keep.sort()
    return [keypoints[i] for i in keep], descriptors[keep]

A 6 × 6 grid at 60 keypoints per cell caps a frame at 2,160 features that are guaranteed to span it. On flights where a third of the frame is water or bare sand, the empty cells simply contribute nothing and the rest of the frame is unaffected — which is exactly the behaviour a global cap fails to give.

# 2. Extracting Features From a Single Frame

Read the first band as a 2D array — src.read() with no index returns a 3D (bands, H, W) array that cv2 rejects — normalise non-8-bit data, then carry the CRS and affine transform alongside the descriptors.

import os
import numpy as np
import rasterio

def extract_one(img_path: str, detector) -> dict | None:
    """Extract keypoints + descriptors from one GeoTIFF, preserving CRS/transform."""
    with rasterio.open(img_path) as src:
        crs = src.crs
        transform = src.transform
        img_data = src.read(1)  # band 1 only -> 2D array for OpenCV

    # Detectors require 8-bit single-channel input.
    if img_data.dtype != np.uint8:
        img_data = cv2.normalize(img_data, None, 0, 255, cv2.NORM_MINMAX).astype(np.uint8)

    kp, desc = detector.detectAndCompute(img_data, None)
    if not kp or desc is None:
        return None

    return {
        "image": os.path.basename(img_path),
        # Persist pt, scale, and orientation so matches can be re-derived later.
        "keypoints": np.array([(p.pt[0], p.pt[1], p.size, p.angle) for p in kp]),
        "descriptors": desc,
        "crs": crs.to_string() if crs else None,
        "transform": transform.to_gdal(),
    }

Figure 3 — A keypoint is four things: a subpixel position, a scale, an orientation, and the descriptor sampled in that rotated, scaled frame. Only the last one dominates memory, which is why the detector choice is a storage decision as much as an accuracy one.

# 3. Batch Processing a Flight Strip

Production runs must be reproducible and memory-bounded. The driver below processes imagery in configurable chunks across a ProcessPoolExecutor, serialises each frame’s results to a compressed .npz immediately, and forces garbage collection between chunks so RAM never climbs monotonically. This mirrors the parallel processing strategies for alignment used elsewhere in the pipeline; keep chunk size and worker count conservative to leave headroom for the descriptor matrices themselves.

import gc
import logging
import concurrent.futures
from pathlib import Path

logging.basicConfig(level=logging.INFO, format="%(asctime)s | %(levelname)s | %(message)s")

def extract_features_chunk(image_paths, detector_type="SIFT", max_features=8000):
    """Worker entry point: build a detector once, process a list of frames."""
    detector = build_detector(detector_type, max_features)
    results = []
    for img_path in image_paths:
        try:
            res = extract_one(img_path, detector)
            if res is not None:
                results.append(res)
        except Exception as e:  # corrupted frame / missing EXIF must not kill the batch
            logging.warning(f"Failed to process {img_path}: {e}")
    return results

def process_flight_strip(image_dir, chunk_size=20, detector="SIFT", output_dir="./features"):
    Path(output_dir).mkdir(parents=True, exist_ok=True)
    images = sorted(str(p) for p in Path(image_dir).glob("*.tif"))
    if not images:
        logging.error("No TIFF images found in directory.")
        return []

    chunks = [images[i:i + chunk_size] for i in range(0, len(images), chunk_size)]
    all_crs = []

    with concurrent.futures.ProcessPoolExecutor(max_workers=min(os.cpu_count(), 4)) as executor:
        futures = [executor.submit(extract_features_chunk, c, detector) for c in chunks]
        for i, future in enumerate(concurrent.futures.as_completed(futures)):
            try:
                for res in future.result():
                    if res["crs"]:
                        all_crs.append(res["crs"])
                    np.savez_compressed(
                        Path(output_dir) / f"{res['image']}.npz",
                        keypoints=res["keypoints"],
                        descriptors=res["descriptors"],
                        crs=res["crs"],
                        transform=res["transform"],
                    )
                logging.info(f"Processed chunk {i + 1}/{len(chunks)}")
            except Exception as e:
                logging.error(f"Chunk processing failed: {e}")
            finally:
                gc.collect()  # release descriptor arrays before the next chunk

    logging.info("Feature extraction complete. Ready for spatial resection.")
    return all_crs

# Persisting Descriptors So a Block Never Sits in RAM

The arithmetic that ends most naive detection scripts is simple. A 600-image block at 4,000 SIFT keypoints per image is 2.4 million descriptors; at 512 bytes each that is 1.2 GB of descriptors alone, before the keypoint structs, before the images, and before the matcher builds its index. Hold all of it and a 16 GB workstation is already in trouble; hold the source rasters too and it is finished.

The fix is not to compress harder but to never accumulate. Each frame’s features are written out the moment they are computed, and later stages read back only the pairs they need. np.savez_compressed is enough: it stores the descriptor matrix, the keypoint array, and the metadata that must travel with them — the CRS string and the affine transform — in one file per frame, keyed by the image name.

Two details make the difference between a store that works and one that quietly poisons the matching stage. The descriptor dtype must be preserved: binary descriptors from ORB and AKAZE are uint8, and a store that promotes them to float32 on the way back in will silently make Hamming distance meaningless. And the keypoint array must keep subpixel precision — rounding pixel coordinates to integers on serialization discards the very thing that makes the reprojection error small, and it is invisible until an accuracy report comes back a few centimetres worse than it should be.

from pathlib import Path

import numpy as np


def save_features(out_dir: Path, image_name: str, keypoints, descriptors,
                  crs_wkt: str, transform) -> Path:
    """One .npz per frame: descriptors, keypoint geometry, and the georeference."""
    kp = np.array([[k.pt[0], k.pt[1], k.size, k.angle, k.response]
                   for k in keypoints], dtype=np.float32)   # subpixel, not rounded
    path = out_dir / f"{Path(image_name).stem}.npz"
    np.savez_compressed(
        path,
        keypoints=kp,
        descriptors=descriptors,          # dtype preserved: uint8 stays uint8
        crs=np.array(crs_wkt),
        transform=np.array(tuple(transform), dtype=np.float64),
    )
    return path

Written this way the peak working set is one frame plus one descriptor matrix, regardless of block size, and the store doubles as a cache: re-running the matching stage after a parameter change costs nothing in re-detection. The same out-of-core discipline governs the point-cloud stages later in the pipeline, described in memory management for large point clouds.

# 4. Enforcing CRS Consistency

Coordinate reference system integrity has to be guarded before descriptors leave this stage. UAV imagery ships with WGS84 (EPSG:4326) or a local projected CRS, and a single mismatched strip propagates into warped orthomosaics, inaccurate DEMs, and failed ground control point registration downstream. Validate that every frame shares one projected CRS, the same discipline applied in managing coordinate reference systems in GDAL.

import pyproj

def validate_crs_consistency(crs_list, project_crs="EPSG:32633") -> list[str]:
    """Confirm every extracted frame shares the target projected CRS."""
    target = pyproj.CRS.from_string(project_crs)
    valid = []
    for crs_str in crs_list:
        if crs_str is None:
            continue
        try:
            source = pyproj.CRS.from_string(crs_str)
            if source.equals(target) or source.to_epsg() == target.to_epsg():
                valid.append(crs_str)
            else:
                logging.warning(f"CRS mismatch: {crs_str} vs target {project_crs}")
        except Exception:
            logging.warning(f"Unparseable CRS string: {crs_str}")
    return valid

# Parameter Deep-Dive

# Verification and Output Inspection

Never hand descriptors to matching without asserting the outputs are well-formed. Check that every frame produced a file, descriptors have the dtype the detector promised (float32 for SIFT, uint8 for ORB/AKAZE), keypoints are non-empty, and the CRS is uniform.

import numpy as np
from pathlib import Path

def inspect_features(output_dir="./features", expected_crs="EPSG:32633"):
    files = sorted(Path(output_dir).glob("*.npz"))
    assert files, "No feature files were written — extraction failed silently."

    crs_seen = set()
    for f in files:
        data = np.load(f, allow_pickle=True)
        kp, desc = data["keypoints"], data["descriptors"]
        assert kp.shape[0] > 0, f"{f.name}: zero keypoints"
        assert desc.shape[0] == kp.shape[0], f"{f.name}: keypoint/descriptor count mismatch"
        crs_seen.add(str(data["crs"]))

    assert crs_seen == {expected_crs}, f"Mixed or unexpected CRS: {crs_seen}"
    print(f"OK: {len(files)} frames, uniform CRS {expected_crs}")

# A median of 2,000–8,000 keypoints per frame is healthy for survey-grade overlap.

If median keypoint counts collapse below a few hundred per frame, the imagery is likely over-smoothed or under-exposed — apply CLAHE (Contrast Limited Adaptive Histogram Equalization) before detection rather than loosening thresholds, which only invites false matches.

# Operational Best Practices

1. Overlap-aware thresholding: for datasets with >80% overlap, cut nfeatures by 30–40%. Redundant tie points inflate matching time without improving geometric stability.
2. Illumination normalisation: apply CLAHE to frames captured during sunrise/sunset transitions to stabilise gradient-based detectors.
3. Descriptor compression: store binary descriptors (ORB, BRISK) in compressed .npz or Parquet; keep SIFT descriptors as float32 to halve I/O.
4. Hardware scaling: on <32 GB RAM, enforce chunk_size ≤ 15 and use numpy.memmap for descriptor caching.
5. Pre-flight metadata gates: flag mixed-CRS datasets or missing focal-length metadata before extraction begins. Validating EXIF GPS data before processing catches most of these early.

# Troubleshooting

AttributeError: module 'cv2' has no attribute 'SIFT_create' You installed opencv-python instead of opencv-contrib-python, or both at once. Uninstall both, then install only opencv-contrib-python. SIFT lives in the contrib build.

detectAndCompute returns desc is None for some frames The frame had no detectable features (uniform water, blown-out sky, or a fully shadowed strip). The extractor already skips these, but a high skip rate means an exposure or thresholding problem — apply CLAHE or lower contrastThreshold slightly rather than ignoring the frames.

cv2.error: (-5:Bad argument) image is empty or has incorrect depth You passed a 3D array. rasterio’s src.read() returns (bands, H, W); use src.read(1) for a 2D band, and normalise non-8-bit data with cv2.normalize(...).astype(np.uint8).

Workers die with MemoryError / the OS OOM-killer terminates the process Each worker holds a full chunk of float32 SIFT descriptors. Reduce max_workers, drop chunk_size to ≤ 15, and lower nfeatures. Confirm headroom before scaling back up.

Matching later produces almost no inliers despite thousands of keypoints Usually a CRS or scale mismatch, not a detection failure. Run validate_crs_consistency and verify GSD is consistent across strips; mismatched projections survive detection but collapse during bundle adjustment optimisation.

Descriptor .npz files load with pickle errors You saved object arrays (e.g. crs=None) and loaded without allow_pickle=True. Add the flag, or store the CRS as a plain string to keep the archive picklable-free.

# Related

• Fixing SIFT vs ORB Performance in UAV Photos
• Optimizing Bundle Adjustment with Python
• Parallel Processing Strategies for Alignment
• Memory Management for Large Point Clouds
• Managing Coordinate Reference Systems in GDAL

← Automated Image Alignment & Feature Matching Workflows