Fixing COG Validation Errors in GDAL

You exported a GeoTIFF you believed was cloud-optimized, ran GDAL’s validate_cloud_optimized_geotiff.py (or rio cogeo validate), and it printed <file> is NOT a valid cloud optimized GeoTIFF followed by a list of concrete faults: The file is not tiled, The file does not have overviews, The offset of the main IFD should be after the offset of the overview IFDs, or Overview block size mismatch. The raster still opens in QGIS and draws correctly, which is exactly why this is confusing — the file is a valid GeoTIFF, it is just not laid out the way a range-request client needs. This page maps each error string to the byte-level cause and gives the minimal re-emit that fixes it deterministically.

# Why GDAL rejects a GeoTIFF as “not a valid COG”

A COG is a GeoTIFF with three extra structural guarantees, and the validator checks each one independently. First, the pixels must be stored in internal tiles (square blocks), not the horizontal strips a default GeoTIFF uses — strips force a client to read a full image-width band to sample any tile. Second, the file must embed decimated overviews, a downsampled pyramid, so a zoomed-out request does not read full-resolution pixels. Third — the subtle one — the IFD ordering must place the overview image file directories and their tile offsets ahead of the main full-resolution image in the byte stream, so the very first HTTP range request lands on the header and pyramid index. A file can satisfy two of these and fail the third; the validator reports exactly which.

The reason a normal write fails ordering is mechanical: GDAL’s classic GTiff driver writes the main image first and appends overviews afterward (or builds them in a sidecar .ovr), so the offsets come out backwards for streaming even when tiling and overviews are both present. The dedicated COG driver exists precisely to reorganize the whole file on close so the offsets are correct. Getting this right is the export stage of the DEM/DSM raster export pipeline, and the full authoring path is covered in exporting Cloud-Optimized GeoTIFFs with Rasterio.

Figure 1 — The validator’s messages map one-to-one onto four structural properties. Reading them as a checklist rather than as errors turns the fix into a lookup.

# Minimal reproducible solution

Almost every validation error is fixed by the same move: re-emit the raster through GDAL’s COG driver, which enforces tiling, builds overviews, and fixes IFD ordering in one pass. The routine below runs the validator, and on failure re-emits with the correct options and re-validates. It is intentionally small so the fix is auditable.

from rio_cogeo.cogeo import cog_translate, cog_validate
from rio_cogeo.profiles import cog_profiles

def repair_cog(src_path: str, dst_path: str, *, is_float: bool = False) -> None:
    """Validate a raster and, if it fails, re-emit a correct COG."""
    valid, errors, warnings = cog_validate(src_path)   # structural check
    if valid:
        print(f"already a valid COG: warnings={warnings}")
        return

    print(f"invalid COG, re-emitting. errors={errors}")
    # 'deflate' base profile: tiled 512, DEFLATE, correct IFD ordering.
    profile = cog_profiles.get("deflate")
    profile.update(blockxsize=512, blockysize=512)
    # predictor 3 for float DEMs, 2 for integer rasters — this is a creation option.
    creation = {"predictor": 3 if is_float else 2}

    cog_translate(
        src_path,
        dst_path,
        profile,
        overview_resampling="average",   # 'nearest' for categorical rasters
        **creation,
    )
    ok, errs, warns = cog_validate(dst_path)            # re-validate the output
    assert ok, f"still invalid after re-emit: {errs}"
    print(f"repaired -> {dst_path}, warnings={warns}")

cog_translate streams the source in windows, so it never loads the whole raster into RAM, and the cog_profiles.get("deflate") base already sets tiled=True with a 512 block — the two conditions that fix the not tiled and block size errors. Passing overview_resampling is what fixes the overviews missing error, and writing through this driver at all is what fixes the IFD ordering error. If you would rather drive this from the command line, gdal_translate -of COG -co COMPRESS=DEFLATE -co PREDICTOR=2 in.tif out.tif produces the identical layout.

Figure 2 — Windowed writing and COG ordering are structurally incompatible: the directory cannot be written first until every tile offset is known. That is why the two-pass export is a design constraint rather than an oversight.

# Edge-case matrix

The exact validator message tells you which guarantee failed. Map it to the cause and the fix.

The block-size mismatch case is the sneaky one: a file can be tiled and have overviews and still fail because the overviews were built at 128 while the main image is 512, so a client cannot use one tiling scheme for both. Re-emitting from source resolution through the COG driver forces both to the same block size.

# Verification snippet

After the repair, assert the three structural guarantees independently so a regression tells you which one broke, not just that validation failed.

import rasterio
from rio_cogeo.cogeo import cog_validate

def assert_valid_cog(path: str) -> None:
    with rasterio.open(path) as ds:
        assert ds.profile["tiled"] is True, "not tiled"
        assert ds.profile["blockxsize"] == 512, "unexpected block size"
        assert len(ds.overviews(1)) >= 3, f"too few overviews: {ds.overviews(1)}"
    ok, errors, _ = cog_validate(path)
    assert ok, f"structural validation failed: {errors}"
    print("COG verified:", ds.overviews(1))

Run this in CI as a hard gate. The overviews(1) list also confirms the pyramid depth — an empty list means the overviews missing error will recur, and fewer than three levels usually means the source was too small or the factors were truncated.

A last check worth adding to CI: validate over HTTP as well as on local disk. The structural rules exist to make range requests work, and a file can satisfy every local check while still being unusable remotely because the server does not honour range requests or strips the relevant headers. Serving the finished COG from a temporary local HTTP server and opening it through a virtual filesystem path exercises the same path a client will, and catches a class of deployment fault that no amount of local validation can see.

# When to escalate

The re-emit above fixes structural (layout) faults. Escalate to the parent guide when the problem is not layout:

• The re-emit succeeds but the file is huge or slow — that is a compression/overview tuning problem, not a validity one. Work through codec, predictor, and pyramid depth in exporting Cloud-Optimized GeoTIFFs with Rasterio and the dedicated setting compression and overviews for orthomosaic COGs page.
• cog_translate itself aborts on memory on a very large mosaic — the layout is fine but the process is exhausting RAM; see decoding GDAL “cannot allocate memory” on export.
• You are choosing between the CLI validator and a scripted check for a whole export stage — the trade-offs live in GDAL vs Rasterio for DEM and orthomosaic export.

Keep the validator itself pinned alongside the rest of the toolchain. The COG specification has been revised since it was first published, and a file that a newer validator accepts can be rejected by an older one running in a client’s environment — or, more awkwardly, the reverse. Recording the validator version next to the pass result turns a disputed file into a two-minute comparison rather than an argument about whose tooling is correct.

# Related

• Exporting Cloud-Optimized GeoTIFFs with Rasterio
• Setting Compression and Overviews for Orthomosaic COGs
• GDAL vs Rasterio for DEM and Orthomosaic Export

← Exporting Cloud-Optimized GeoTIFFs with Rasterio