Bulk Rewriting EXIF Without Recompressing JPEGs

PPK processing has produced corrected camera positions for a survey, and they need to go back into the images so the reconstruction picks them up. The obvious approach — open each image, set the tags, save — quietly re-encodes every JPEG, costing a generation of quality on ten thousand files and several hours of processing.

Metadata lives in segments that sit alongside the compressed image data, and editing it does not require touching the pixels at all. This page covers doing that correctly at survey scale: which tools edit losslessly, how to write atomically so an interrupted run does not corrupt a dataset, and how to verify that the pixels really were untouched. It completes the metadata work in validating and repairing EXIF metadata at scale.

# Why a naive rewrite is expensive

A JPEG is a sequence of segments: metadata segments, a quantisation table, Huffman tables, and the entropy-coded image data. Changing a metadata segment means rewriting that segment and copying the rest byte for byte — an operation that takes microseconds and alters nothing about the image.

An imaging library that opens a JPEG decodes it to pixels, and saving re-encodes them. That costs time, costs a compression generation, and on a survey introduces artefacts that will show up in the reconstruction’s feature matching. It also silently changes the file’s quantisation tables to the library’s defaults, so the delivered imagery no longer matches what the camera produced.

The difference is not marginal. On ten thousand 20-megapixel images, a metadata-only rewrite is a few minutes and a re-encode is a few hours plus a measurable loss of detail.

Figure 1 — Two paths, one of which never touches a pixel.

# Minimal reproducible solution

ExifTool edits metadata in place without decoding, and its batch mode handles a survey in one process.

import csv
import subprocess
from pathlib import Path


def write_positions(updates: list[dict], *, backup: bool = True) -> dict:
    """Write corrected GPS positions to many images in one ExifTool call.

    A CSV-driven update is the fastest safe form: ExifTool reads the file,
    matches on SourceFile, and applies each row. One process handles ten
    thousand images in minutes, where one process per image would take hours
    in startup cost alone.
    """
    csv_path = Path("exif_updates.csv")
    fields = ["SourceFile", "GPSLatitude", "GPSLongitude", "GPSAltitude",
              "GPSLatitudeRef", "GPSLongitudeRef", "GPSAltitudeRef"]
    with csv_path.open("w", newline="") as fh:
        writer = csv.DictWriter(fh, fieldnames=fields)
        writer.writeheader()
        for row in updates:
            writer.writerow({k: row.get(k, "") for k in fields})

    cmd = ["exiftool", "-csv=" + str(csv_path), "-preserve"]
    if not backup:
        cmd.append("-overwrite_original")
    cmd.append(str(Path(updates[0]["SourceFile"]).parent))

    result = subprocess.run(cmd, capture_output=True, text=True)
    if result.returncode != 0:
        raise RuntimeError(f"exiftool failed: {result.stderr[:400]}")
    return {"updated": len(updates), "stdout": result.stdout.strip()[-200:]}

-preserve keeps the file’s modification time, which matters more than it sounds: a pipeline that sorts by file time to recover capture order will otherwise find every image stamped with the moment of the rewrite.

-overwrite_original is the flag to think about. Without it ExifTool keeps a _original copy of every file, which doubles the storage for a survey; with it, an interrupted run leaves some images updated and some not, with no record of which.

# Writing atomically

For a survey that matters, neither of those is acceptable, and the answer is to write to a new directory and swap.

import shutil
import subprocess
from pathlib import Path


def rewrite_to_new_directory(src_dir: str, dst_dir: str, csv_path: str) -> dict:
    """Copy, then edit the copy, so the originals are never at risk.

    ExifTool's -o option writes new files rather than editing in place, so an
    interrupted run leaves a partial output directory and an untouched input
    — which is recoverable by simply running it again.
    """
    src, dst = Path(src_dir), Path(dst_dir)
    if dst.exists() and any(dst.iterdir()):
        raise FileExistsError(f"{dst} is not empty; refusing to write into it")
    dst.mkdir(parents=True, exist_ok=True)

    result = subprocess.run(
        ["exiftool", f"-csv={csv_path}", "-preserve", "-o", str(dst) + "/", str(src)],
        capture_output=True, text=True)
    if result.returncode != 0:
        raise RuntimeError(f"exiftool failed: {result.stderr[:400]}")

    src_count = len(list(src.glob("*.JPG"))) + len(list(src.glob("*.jpg")))
    dst_count = len(list(dst.glob("*.JPG"))) + len(list(dst.glob("*.jpg")))
    if src_count != dst_count:
        raise RuntimeError(f"wrote {dst_count} of {src_count} images — incomplete run")
    return {"images": dst_count, "source": str(src), "output": str(dst)}

Figure 3 — The distinction is whether the pixel data is touched at all.

# Edge-case matrix

The hemisphere row causes the most spectacular failures. GPSLatitude is written as an unsigned value with a separate GPSLatitudeRef of N or S, so an update that sets the value and not the reference places a southern-hemisphere survey in the north. The mirrored result is unmistakable and entirely avoidable.

def gps_tags(lat: float, lon: float, alt: float) -> dict:
    """Position tags with their reference tags, which must never be omitted."""
    return {
        "GPSLatitude": abs(lat), "GPSLatitudeRef": "N" if lat >= 0 else "S",
        "GPSLongitude": abs(lon), "GPSLongitudeRef": "E" if lon >= 0 else "W",
        "GPSAltitude": abs(alt), "GPSAltitudeRef": 0 if alt >= 0 else 1,
    }

# Verification snippet

Verification has two parts: the metadata is what was intended, and the pixels are untouched.

import hashlib
import subprocess
from pathlib import Path


def image_data_digest(path: str) -> str:
    """Hash only the compressed image data, ignoring every metadata segment.

    Two files with identical image data and different metadata produce the
    same digest, which is exactly the property needed to prove a rewrite was
    lossless.
    """
    raw = subprocess.run(["exiftool", "-b", "-JpgFromRaw", "-ThumbnailImage", path],
                         capture_output=True).stdout
    data = Path(path).read_bytes()
    start = data.find(b"\xff\xda")            # start of scan
    if start < 0:
        raise ValueError(f"{path}: no scan marker found")
    return hashlib.sha256(data[start:]).hexdigest()[:16]


def verify_rewrite(originals: list[str], rewritten: list[str],
                   expected: dict[str, dict]) -> dict:
    """Pixels unchanged, metadata as intended."""
    problems = []
    for a, b in zip(sorted(originals), sorted(rewritten)):
        if image_data_digest(a) != image_data_digest(b):
            problems.append(f"{Path(b).name}: image data changed — this was a re-encode")
    out = subprocess.run(["exiftool", "-j", "-GPSLatitude", "-GPSLatitudeRef",
                          *rewritten[:50]], capture_output=True, text=True)
    return {"checked": len(rewritten), "problems": problems,
            "sample_metadata": out.stdout[:400]}

The scan-marker digest is the definitive test. It hashes the entropy-coded data and nothing else, so it is unaffected by any metadata change and changes immediately if a pixel was touched.

Figure 2 — What each strategy leaves behind when a run does not finish.

# Whether to rewrite at all

Before writing anything, it is worth asking whether the positions need to be in the images. Several reconstruction tools accept an external position file, which avoids touching the imagery entirely and keeps the originals as the single source of truth.

Writing into the images wins when the imagery will be handed to somebody else’s software, or archived and reprocessed later by a tool whose position-file format is unknown. It loses when the pipeline is self-contained, because every rewrite is an opportunity for a partial run, a hemisphere error or an accidental re-encode.

The middle position — keep the originals untouched, write a corrected copy for delivery, and record the position file alongside both — costs storage and removes every failure mode on this page from the archive. On a survey programme where the raw imagery is kept anyway, that is usually the right trade.

# When to escalate

• The source media is read-only or nearly full. Copy to working storage first. An interrupted write on full media is the one case that can corrupt files.
• Both EXIF and XMP carry positions. Update both or strip the stale one, or a downstream reader that prefers XMP will use the old values.
• A re-encode has already happened. The lost detail is not recoverable. Re-derive from the originals if they exist, and add a digest check to the pipeline so it cannot recur.

# Related

• Validating and Repairing EXIF Metadata at Scale
• Applying PPK Corrections to Image Timestamps
• Repairing Corrupt or Truncated EXIF Headers

← Validating and Repairing EXIF Metadata at Scale