Repairing Corrupt or Truncated EXIF Headers

The ingest run reports forty frames with no GPS. They open fine in an image viewer, they look correct, and every one of them was shot in the middle of a flight where the frames on either side are perfect. The card filled, or the camera lost power, or a transfer was interrupted — and the result is a set of files whose pixel data survived and whose metadata did not.

This page classifies the damage, recovers the frames that can be recovered, and quarantines the rest with a reason attached, so that a partially damaged batch produces a smaller good dataset rather than a silently biased one.

# Why the pixels survive and the metadata does not

A JPEG is a sequence of segments. The EXIF block sits near the front, in an APP1 segment, and the compressed image data follows. That layout produces two asymmetric failure modes.

Truncation at the tail removes image data and leaves the header intact. The decoder produces a partial image — grey below some scanline — and every EXIF tag reads correctly. These frames have valid metadata and unusable pixels.

Corruption in the header damages the APP1 segment while the image data is untouched. The decoder skips the malformed segment and renders the frame perfectly; EXIF parsing returns nothing, or returns a subset. These frames have usable pixels and no position.

The second case is the one that reaches a reconstruction, because nothing about the image looks wrong. A pipeline that drops frames without GPS silently discards them; one that does not check ends up feeding the solver a block with a hole in its position coverage.

Figure 1 — The layout that makes the two failures asymmetric. A viewer renders the dangerous case flawlessly, because rendering is exactly the operation that does not consult EXIF.

# Minimal reproducible solution

Classification comes first: read the structure rather than trusting a parser’s exception. Four states are distinguishable from the bytes.

from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Diagnosis:
    path: Path
    state: str       # ok | truncated | no_exif | not_jpeg
    detail: str


def classify(path: Path) -> Diagnosis:
    """Structural classification from the file's own markers."""
    data = path.read_bytes()
    if len(data) < 4 or data[:2] != b"\xff\xd8":
        return Diagnosis(path, "not_jpeg", "missing start-of-image marker")

    if data[-2:] != b"\xff\xd9":
        return Diagnosis(path, "truncated",
                         f"no end-of-image marker; {len(data):,} bytes")

    # APP1 immediately follows SOI on a normal camera JPEG.
    if data[2:4] != b"\xff\xe1":
        return Diagnosis(path, "no_exif", "no APP1 segment after SOI")

    seg_len = int.from_bytes(data[4:6], "big")
    if data[6:12] != b"Exif\x00\x00":
        return Diagnosis(path, "no_exif", "APP1 present but not EXIF")
    if 2 + seg_len > len(data):
        return Diagnosis(path, "no_exif",
                         f"APP1 claims {seg_len} bytes, file has {len(data) - 2}")
    return Diagnosis(path, "ok", f"APP1 {seg_len} bytes")

Checking the declared segment length against the file size catches the case a parser reports as a generic error: a header whose length field was written before the write failed, so it describes a segment larger than the file that contains it.

With the classification in hand, recovery is possible in exactly one of the four states. A frame with a damaged APP1 but intact pixels can be given metadata from its neighbours — not its position, which must not be invented, but its camera model, focal length and orientation, which are constant across a flight and which the reconstruction needs in order to use the frame at all.

import piexif


def restore_camera_tags(damaged: Path, donor: Path, out: Path) -> None:
    """Copy the invariant camera tags from a good neighbouring frame.

    Deliberately does NOT copy GPS or timestamps: those are per-frame
    measurements, and a borrowed one is a fabricated observation.
    """
    good = piexif.load(str(donor))
    keep_0th = {piexif.ImageIFD.Make, piexif.ImageIFD.Model,
                piexif.ImageIFD.Orientation}
    keep_exif = {piexif.ExifIFD.FocalLength, piexif.ExifIFD.FocalLengthIn35mmFilm,
                 piexif.ExifIFD.PixelXDimension, piexif.ExifIFD.PixelYDimension}

    rebuilt = {
        "0th": {k: v for k, v in good["0th"].items() if k in keep_0th},
        "Exif": {k: v for k, v in good["Exif"].items() if k in keep_exif},
        "GPS": {},                       # left empty on purpose
        "1st": {}, "thumbnail": None,
    }
    piexif.insert(piexif.dump(rebuilt), str(damaged), str(out))

Leaving GPS empty is the important half. A frame with borrowed camera parameters and no position is a usable image the solver will place by feature matching. A frame with a borrowed position is a false observation that will pull the block, and it is indistinguishable from a real one once written.

# Edge-case matrix

The sixth row is why structural classification is necessary but not sufficient: a file can be structurally perfect and contain a GPS sub-directory full of nonsense. That is caught downstream by the plausibility checks in how to validate EXIF GPS data before processing, and the two checks are complementary rather than alternatives.

Figure 2 — The routing. Two of the three destinations still contribute imagery; what matters is that the third is counted rather than merely absent.

# Verification snippet

After repair, the frames that were fixed must decode and must still lack GPS — the second half being the part that is easy to get wrong by copying a whole EXIF block.

import piexif
from PIL import Image


def assert_repair_sound(path) -> None:
    """A repaired frame decodes fully and carries no borrowed position."""
    with Image.open(path) as im:
        im.load()                            # raises on truncated data
        assert im.size[0] > 0 and im.size[1] > 0

    tags = piexif.load(str(path))
    assert tags["0th"].get(piexif.ImageIFD.Model), "camera model not restored"
    assert tags["Exif"].get(piexif.ExifIFD.FocalLength), "focal length not restored"
    assert not tags["GPS"], (
        "repaired frame carries GPS tags — a borrowed position is a fabricated "
        "observation and must not be written")

im.load() rather than Image.open() alone is what makes the decode check real: opening a JPEG reads only the header, and a truncated file opens without complaint. Forcing the decode is what surfaces the missing scanlines.

One further guard belongs in the ingest step rather than in the repair. Frames that lose their position but keep their pixels create a coverage gap in the position graph even when the imagery is complete, and on a block that relies on camera priors for its initial poses that gap can be the difference between a converging reconstruction and one that splits. Counting positionless frames per flight strip, rather than per batch, is what makes such a gap visible: forty frames without GPS spread evenly across a survey are a nuisance, while forty consecutive ones are a hole in the trajectory that the solver will feel.

# When to escalate

• More than a few per cent of a batch is damaged. This is a hardware signal rather than a per-file problem. A card producing scattered truncations is failing, and the correct response is to stop using it and re-copy from the original if one exists, not to repair forty frames and continue.
• Damage is contiguous in capture order. A run of consecutive damaged frames is a write interruption — a battery event or a card swap — and the frames are usually genuinely lost. The coverage gap that leaves is a flight-geometry problem, evaluated against the overlap floor rather than fixed here.
• Frames repair cleanly and the reconstruction still drops them. The pixels are intact and something else disqualifies them, most often a dimension or camera model that does not match the rest of the block. That is the mixed-sensor case described in handling mixed sensor data in photogrammetry pipelines.

# Related

• Troubleshooting ingestion and CRS failures
• How to validate EXIF GPS data before processing
• Best practices for storing raw UAV datasets

← Troubleshooting Ingestion and CRS Failures

Figure 3 — The cheapest fix is upstream of this page entirely. Repair recovers metadata; nothing recovers pixels that were never written.