Diagnosing Empty Output from a PDAL Pipeline

pdal pipeline exits zero. The output file exists, is 400 bytes, and contains no points. Nothing in the log says anything is wrong, because from PDAL’s point of view nothing is: a filter that removes every point has done exactly what it was asked, and a writer that receives no points writes a valid empty file.

This is the most common “silent” failure in point-cloud work, and it is also one of the quickest to diagnose once the pipeline is treated as a sequence to be bisected rather than a unit that succeeded or failed.

The five stages that empty a cloud

A range filter whose limits exclude everything. Classification[2:2] on a cloud that was never classified matches nothing. So does Z[100:200] on a site at 340 m. The syntax is easy to get subtly wrong too: Classification![7:7] means “not 7” and Classification[7:7] means “only 7”, and transposing them removes exactly the points you meant to keep.

A crop with bounds in the wrong CRS. A bounding box in latitude and longitude applied to a cloud in UTM selects a region a few metres across near the origin, which contains nothing. The filter is behaving correctly; the bounds are in a different space.

A classification-dependent filter before the classifier. Ordering matters, and PDAL runs stages in the order written. Filtering to ground before the ground filter runs yields nothing.

An expression referring to a dimension that is all zero. Extra dimensions default to zero when created, so a filter on a computed value that was never actually computed passes nothing.

A reprojection that moved the data out of a later filter’s bounds. Reprojecting and then cropping with bounds from the source CRS is the same failure as the second one, arrived at from the other direction.

Point count through a pipeline that silently empties A bar chart of point count after each of five pipeline stages. The reader yields two hundred and fourteen million points, the assign stage the same, the reprojection the same, then a crop stage drops to zero and the writer produces zero. An annotation marks the crop as the offending stage and notes that its bounds were in latitude and longitude while the data had just been reprojected to UTM. A note states that the pipeline exited with success because no stage encountered an error. reader assign reproject crop writer 214 M 214 M 214 M 0 0 bounds were in degrees after a reprojection to UTM Exit status zero. Every stage did exactly what it was told. One number per stage turns "the output is empty" into "stage four emptied it".

Figure 1 — The diagnosis is a single chart, and producing it takes one function.

Minimal reproducible solution

Bisect by running truncated pipelines and recording the count after each stage.

import json
import subprocess


def counts_by_stage(pipeline: dict) -> list[dict]:
    """Point count after each stage of a pipeline, stopping at the first zero.

    Each truncated pipeline gets a stats filter appended so PDAL reports a
    count without writing anything. It costs one read per stage, which is
    acceptable for a diagnosis and far cheaper than guessing.
    """
    stages = pipeline["pipeline"]
    out = []
    for i in range(1, len(stages) + 1):
        stage = stages[i - 1]
        label = stage.get("type", "reader") if isinstance(stage, dict) else "reader"
        partial = {"pipeline": stages[:i] + [{"type": "filters.stats",
                                              "dimensions": "X"}]}
        proc = subprocess.run(
            ["pdal", "pipeline", "--stdin", "--metadata=/dev/stdout"],
            input=json.dumps(partial), text=True, capture_output=True)
        try:
            meta = json.loads(proc.stdout)
            n = int(meta["stages"]["filters.stats"]["statistic"][0]["count"])
        except Exception:
            n = None
        out.append({"index": i, "stage": label, "points": n})
        if n == 0:
            break
    return out

For a large cloud, bisecting properly — testing the middle stage first and halving — is worth the extra bookkeeping. For anything under a hundred million points, running every prefix is simpler and fast enough.

A guard that fails where the problem is

Better than diagnosing after the fact is failing at the stage that emptied the cloud. PDAL does not do this itself, but a stats filter with an assertion after each suspect stage achieves it.

import json
import subprocess


def guarded(pipeline_stages: list, *, guard_after: set[str]) -> list:
    """Insert a count assertion after every stage that can empty a cloud.

    filters.expression with a condition that is always true does nothing to
    the data but forces the stage to execute, and the stats filter after it
    gives a count the caller can assert on.
    """
    out = []
    for stage in pipeline_stages:
        out.append(stage)
        stage_type = stage.get("type") if isinstance(stage, dict) else None
        if stage_type in guard_after:
            out.append({"type": "filters.stats", "dimensions": "X",
                        "tag": f"count_after_{stage_type.replace('.', '_')}"})
    return out


EMPTYING_STAGES = {"filters.range", "filters.crop", "filters.expression",
                   "filters.decimation", "filters.sample"}

The tags are what make the resulting metadata readable: instead of an anonymous list of statistics, the output names which stage each count came after.

Where a pipeline's points disappear, checked in order A four-stage bisection. Stage one counts points at the reader, confirming the file was read at all. Stage two counts after any spatial filter, where a bounds expressed in the wrong coordinate reference system removes everything. Stage three counts after any attribute filter, where a classification or return-number condition that matches nothing empties the stream. Stage four counts at the writer, where an output already open or a path that cannot be created produces a zero-length file despite points arriving. A note states that inserting a counting stage between every filter finds the culprit in one run. 1. at the reader was the file read at all 2. after spatial filter bounds in the wrong coordinate system 3. after attribute filter a condition that matches nothing 4. at the writer path unwritable despite points arriving A counting stage between every filter finds the culprit in a single run.

Figure 3 — Bisect the pipeline rather than reasoning about it.

Edge-case matrix

Cause Signature Fix
Range limits exclude everything Zero after a range filter Check the limit syntax and the actual value ranges
! inverted on a range Zero, or the exact complement of what you wanted [7:7] is “only 7”; ![7:7] is “not 7”
Crop bounds in the wrong CRS Zero after a crop Express bounds in the CRS at that point in the chain
Crop after reprojection Zero after a crop Reorder, or reproject the bounds too
Class filter before classification Zero after the class filter Reorder the stages
Filter on an uncomputed dimension Zero, dimension is all zero Confirm the creating stage runs first
Empty input file Zero at the reader Check the source before blaming the pipeline
Reader glob matching nothing Zero at the reader, no error Assert the file list is non-empty before building the pipeline

The reader-glob row is worth guarding explicitly, because it produces the same empty output from a completely different cause and is easy to overlook when the pipeline itself looks wrong:

from pathlib import Path


def assert_inputs(pattern: str) -> list[str]:
    """Fail before building a pipeline if the input glob matches nothing."""
    paths = sorted(str(p) for p in Path().glob(pattern))
    if not paths:
        raise FileNotFoundError(f"no files match {pattern!r} — the pipeline would "
                                "produce an empty result with no error")
    return paths

Verification snippet

import json
import subprocess


def assert_non_empty(path: str, *, minimum: int = 1) -> int:
    """Refuse to accept an output file that contains no points."""
    proc = subprocess.run(["pdal", "info", "--summary", path],
                          capture_output=True, text=True, check=True)
    n = int(json.loads(proc.stdout)["summary"]["num_points"])
    if n < minimum:
        raise ValueError(f"{path} contains {n} points — the pipeline emptied the cloud")
    return n

Putting this immediately after every pipeline run is a two-line change that converts every instance of this failure from a silent one into an immediate one. It does not tell you which stage was responsible, but it stops the empty file reaching anything downstream — and the bisection above is a minute away when it does fire.

Range filter syntax and what each form selects Four range expressions with the point populations they select, drawn as a classification axis from zero to seven. Classification bracket two colon two selects only ground. Classification exclamation bracket seven colon seven selects everything except noise. Classification bracket two colon six selects ground through building inclusive. Classification exclamation bracket two colon two selects everything except ground, which is the expression most often written by mistake when the intent was to remove noise. A note states that all four are valid and none of them errors. classification value 012 345 67 Classification[2:2] — only ground Classification![7:7] — everything except noise Classification[2:6] — ground through building Classification![2:2] — everything except ground The fourth is the one written by mistake when the intent was to drop noise. All four are valid syntax. None of them errors. Only the counts distinguish them.

Figure 2 — Four expressions, four populations. The failure is never a syntax error.

Preventing it rather than diagnosing it

Three habits remove most instances of this failure before it can happen.

Express filter conditions against values you have seen. Before writing Z[100:200], run pdal info --stats and look at the actual range. A minute spent checking the distribution avoids a filter that is correct in intent and wrong in numbers, and it is the same minute whether the pipeline is new or inherited.

Keep CRS changes and spatial filters apart. Where a pipeline both reprojects and crops, write the crop before the reprojection and express its bounds in the source CRS, or write it after and express them in the target. Mixing the two — bounds from a GIS session in one CRS applied after a reprojection to another — is the single most common instance of this failure, and the ordering convention removes the ambiguity entirely.

Make stage order explicit in the code that builds the pipeline. A pipeline assembled by appending stages in whatever order the calling code happens to run is one where a classification-dependent filter can drift ahead of the classifier during a refactor. Building the stage list from a named sequence, with the dependencies asserted, costs a few lines and makes the ordering a property of the design rather than of the call site.

DEPENDS_ON = {
    "filters.hag_nn": set(),
    "filters.smrf": set(),
    "filters.covariancefeatures": set(),
}
PRODUCES = {"filters.smrf": {"Classification"},
            "filters.hag_nn": {"HeightAboveGround"},
            "filters.covariancefeatures": {"Planarity", "Linearity"}}


def assert_stage_order(stages: list[dict]) -> None:
    """Fail at build time when a stage consumes a dimension nothing has created."""
    available: set[str] = {"X", "Y", "Z", "Intensity", "Classification"}
    for stage in stages:
        if not isinstance(stage, dict):
            continue
        needed = set()
        for value in (stage.get("limits", ""), str(stage.get("value", ""))):
            for dim in ("HeightAboveGround", "Planarity", "Linearity", "ExcessGreen"):
                if dim in str(value):
                    needed.add(dim)
        missing = needed - available
        if missing:
            raise ValueError(f"{stage.get('type')} uses {sorted(missing)} "
                             "before any stage creates it")
        available |= PRODUCES.get(stage.get("type", ""), set())

When to escalate

  • The input file itself is empty or truncated. Look upstream at the reconstruction rather than at the pipeline; an incomplete write from a previous stage will do this.
  • The counts are non-zero throughout and the writer still produces nothing. Check the writer’s own options — a where clause on the writer, or a bounds option, can filter at the last step.
  • The pipeline empties only on some inputs. The filter is correct and some inputs genuinely contain no matching points. That is a data finding, and the right response is to record it rather than to widen the filter.

← Troubleshooting Point Cloud Processing Failures