Reading E57 and PLY into a Python Pipeline

The surveyor sends a terrestrial scan of the quarry face to merge with the drone data. It arrives as a 6 GB E57, and the first attempt to read it produces a cloud sitting at the origin with the scanner’s own coordinate frame, in millimetres, with no colour. The second attempt, after converting through a mesh tool to PLY, produces something positioned correctly and missing the intensity the surveyor said was there.

Neither format is broken. Both carry conventions that differ from LAS in ways that a naive read silently discards, and knowing which conventions they are makes the ingest routine rather than exploratory. This page covers bringing both into a pipeline built around LAS, as described in point cloud formats and interchange in Python.

What each format assumes

E57 is a container for one or more scans, each with its own local coordinate frame and a pose — a translation and a quaternion rotation — that places it in a shared frame. Reading the point data without applying the pose gives every scan stacked at the origin. There is also no requirement that the shared frame is georeferenced at all: many E57 files are in an arbitrary project frame, and the transformation to a real CRS lives in the surveyor’s notes rather than the file.

Units are a second trap. E57 nominally uses metres, but scanner exports commonly write millimetres and declare it in the metadata, which a reader that ignores metadata will not apply.

PLY is a mesh format that happens to store points. It has no concept of a coordinate reference system at all, stores coordinates as 32-bit floats by default — which quantises projected coordinates exactly as glTF does — and defines its own per-vertex properties with arbitrary names. A PLY from one tool calls the colour channels red, green, blue; from another, diffuse_red and friends.

Scan poses in an E57 file, applied and unapplied Two plan views. On the left, three terrestrial scans are shown stacked on top of each other at the origin because their poses were not applied; each scan is internally correct and they are mutually meaningless. On the right, the same three scans are placed around a quarry face by their recorded translations and rotations, forming one coherent cloud. Below, a note records that the shared frame may still be an arbitrary project frame rather than a projected coordinate reference system, and that the transformation to a real CRS is often not in the file at all. poses not applied poses applied three scans, all at the origin one coherent cloud in the shared frame the shared frame is not necessarily a CRS that transformation is often in the surveyor's notes, not in the file

Figure 1 — The first thing to check in any E57: whether the poses were applied, and what frame they place the scans in.

Minimal reproducible solution

import numpy as np
import pye57


def read_e57(path: str, *, scan_index: int | None = None) -> dict:
    """Read an E57 into arrays, applying scan poses and unit scaling.

    pye57 applies the pose when `transform=True`, which is not the default in
    every version — passing it explicitly is cheaper than debugging a cloud
    stacked at the origin.
    """
    f = pye57.E57(path)
    count = f.scan_count
    indices = range(count) if scan_index is None else [scan_index]

    chunks, colours, intensities = [], [], []
    for i in indices:
        data = f.read_scan(i, intensity=True, colors=True,
                           row_column=False, transform=True)
        xyz = np.column_stack([data["cartesianX"], data["cartesianY"],
                               data["cartesianZ"]])
        chunks.append(xyz)
        if "colorRed" in data:
            colours.append(np.column_stack([data["colorRed"], data["colorGreen"],
                                            data["colorBlue"]]))
        if "intensity" in data:
            intensities.append(np.asarray(data["intensity"]))

    xyz = np.vstack(chunks)
    header = f.get_header(indices[0] if indices else 0)
    return {
        "xyz": xyz,
        "rgb": np.vstack(colours) if colours else None,
        "intensity": np.concatenate(intensities) if intensities else None,
        "scans": count,
        # Reported so the caller can convert; E57 exports in millimetres exist.
        "declared_units": getattr(header, "lengthUnit", "unknown"),
        "bounds": (xyz.min(axis=0).tolist(), xyz.max(axis=0).tolist()),
    }

Reporting the declared units and the bounds rather than silently converting is deliberate. A cloud whose bounds span 40,000 in each axis is either a 40 km survey or a 40 m scan in millimetres, and only the caller knows which is plausible for the job.

For PLY, the work is in normalising property names and promoting the coordinates out of float32.

import numpy as np
from plyfile import PlyData

COLOUR_ALIASES = {
    "red": "red", "green": "green", "blue": "blue",
    "diffuse_red": "red", "diffuse_green": "green", "diffuse_blue": "blue",
    "r": "red", "g": "green", "b": "blue",
}


def read_ply(path: str) -> dict:
    """Read a PLY point cloud, normalising colour property names.

    Coordinates are promoted to float64 immediately. PLY commonly stores them
    as float32, which quantises a projected easting to a few centimetres, and
    every operation after the read inherits that.
    """
    ply = PlyData.read(path)
    v = ply["vertex"].data
    names = set(v.dtype.names)

    xyz = np.column_stack([v["x"], v["y"], v["z"]]).astype(np.float64)

    channels = {}
    for src, canonical in COLOUR_ALIASES.items():
        if src in names:
            channels[canonical] = np.asarray(v[src])
    rgb = (np.column_stack([channels["red"], channels["green"], channels["blue"]])
           if {"red", "green", "blue"} <= set(channels) else None)

    return {"xyz": xyz, "rgb": rgb,
            "properties": sorted(names),
            "coordinate_dtype": str(v["x"].dtype),
            "precision_warning": ("float32 coordinates — up to a few centimetres "
                                  "of quantisation on projected values"
                                  if v["x"].dtype == np.float32 else None)}
What survives and what is lost converting E57 or PLY into LAS Three rows. Surviving in both directions are coordinates, intensity where present, and colour, which map cleanly onto LAS fields. Lost from E57 are per-scan pose, the scanner's own coordinate frame, embedded panoramic imagery and per-scan metadata, none of which LAS has a place for. Lost from PLY are arbitrary custom per-point attributes, which have no LAS equivalent unless they are mapped explicitly onto extra bytes. A note advises recording what was dropped in the conversion rather than discovering it when somebody asks for it back. survives both ways coordinates, intensity, colour — these map cleanly onto LAS fields lost from E57 per-scan pose, scanner frame, panoramic imagery, per-scan metadata lost from PLY arbitrary custom attributes, unless mapped explicitly onto extra bytes Record what the conversion dropped, rather than discovering it when somebody asks for it back.

Figure 3 — Conversion into LAS is lossy in a specific, enumerable way.

Edge-case matrix

Input variant Naive result Correct handling
Multi-scan E57, poses unapplied Scans stacked at the origin Read with the transform applied
E57 in millimetres Cloud 1000× too large Read the declared unit and scale
E57 in an arbitrary project frame Positioned wrongly, plausibly Obtain the transformation separately
E57 with intensity but no colour Grey cloud Map intensity to a display channel if needed
PLY float32 coordinates Centimetre quantisation Promote to float64; ask for a re-export if precision matters
PLY with diffuse_red naming No colour found Normalise property aliases
PLY with 0–1 float colours Black cloud after ×257 Detect the range before scaling
Either format, no CRS Opens at the origin downstream Assign the CRS explicitly at ingest

The 0–1 colour case is worth a guard, because it silently produces a black cloud through the LAS 16-bit conversion:

import numpy as np


def to_las_colour(rgb: np.ndarray) -> np.ndarray:
    """Normalise any common colour encoding to LAS's unsigned 16-bit range."""
    arr = np.asarray(rgb)
    peak = float(arr.max()) if arr.size else 0.0
    if arr.dtype.kind == "f" and peak <= 1.0:
        scaled = arr * 65535.0                  # 0–1 floats
    elif peak <= 255.0:
        scaled = arr.astype(np.float64) * 257.0  # 8-bit
    else:
        scaled = arr.astype(np.float64)          # already 16-bit
    return np.clip(scaled, 0, 65535).astype(np.uint16)

Verification snippet

import numpy as np


def sanity_check_import(xyz: np.ndarray, expected_extent_m: float,
                        expected_centre: tuple[float, float] | None = None,
                        tol_factor: float = 5.0) -> dict:
    """Does the imported cloud sit where and at the scale the job expects?"""
    lo, hi = xyz.min(axis=0), xyz.max(axis=0)
    extent = float(np.max(hi - lo))
    problems = []

    ratio = extent / expected_extent_m
    if ratio > tol_factor:
        problems.append(f"extent {extent:.0f} is {ratio:.0f}× the expected "
                        f"{expected_extent_m:.0f} m — units are probably millimetres")
    if ratio < 1 / tol_factor:
        problems.append(f"extent {extent:.1f} is far smaller than expected — "
                        "scan poses may not have been applied")
    if expected_centre is not None:
        centre = ((lo + hi) / 2)[:2]
        offset = float(np.hypot(*(centre - np.asarray(expected_centre))))
        if offset > 10 * expected_extent_m:
            problems.append(f"centre is {offset:.0f} m from the expected position — "
                            "wrong coordinate frame")
    return {"extent_m": extent, "bounds": [lo.tolist(), hi.tolist()],
            "problems": problems}

The extent ratio check catches both the millimetre-units and the unapplied-poses cases with one comparison, and it needs nothing beyond a rough idea of how big the site is — which the job definition always has.

Ingest checks that run before a foreign cloud joins the pipeline A linear sequence of five gates a foreign point cloud passes through on import. First the units are checked against the expected site extent. Second the coordinate frame is checked against an expected centre. Third the colour encoding is detected and normalised. Fourth a coordinate reference system is assigned explicitly, because neither format reliably carries one. Fifth the cloud is written to LAS with a derived header. Each gate names the failure it catches, and a note records that all five are cheap and none of them is optional. units mm vs m frame poses applied? colour 0–1, 8-bit, 16-bit CRS assign explicitly write LAS derived header Five gates, none optional, all cheap. Each catches a failure that produces a plausible-looking cloud rather than an error. record the source file and every assumption made an assumed CRS is the one thing nobody remembers a month later

Figure 2 — The ingest gate for foreign clouds. Everything it catches produces plausible output rather than a crash.

Merging a terrestrial scan with drone data

Bringing the two together is a registration problem, not a format one, and it is worth separating the two steps explicitly. Import and normalise the scan first, verify it independently, and only then register it against the drone cloud using the stable-ground approach in aligning two epochs with ICP before differencing.

Two differences from an epoch-to-epoch registration matter. A terrestrial scan is usually far denser and more accurate than the drone cloud in its own small area, so the scan should be the reference and the drone data the moving set — the opposite of the instinct to “fit the new data to the existing model”. And the overlap between them is often a thin band where the drone saw the top of a face and the scanner saw its front, which is poorly conditioned for a rigid fit; adding targets visible to both instruments is far more reliable than relying on surface geometry.

Record the resulting transform with the merged product. A combined cloud whose provenance is “these two files, registered with this transform, on this date” can be rebuilt; one that exists only as a merged file cannot be corrected when either input is revised.

When to escalate

  • The E57 is in an arbitrary project frame with no georeferencing. No amount of reading fixes that. The surveyor holds the transformation, and guessing it from surface fitting introduces error nobody can quantify later.
  • PLY float32 coordinates and the job needs millimetres. Ask for a re-export in a format that carries double precision. The quantisation is already in the file.
  • Colour or intensity is missing entirely. Check whether the export omitted it rather than assuming the scan lacks it; most scanner exports have an options dialog where these are off by default.

← Point Cloud Formats and Interchange in Python