#FlatGeobuf Streaming Reads over HTTP Range Requests

A bounding-box query against a FlatGeobuf file in object storage costs two or three HTTP range reads, no server, and no decompression — because the file carries a packed Hilbert R-tree whose node offsets are computable, and stores features in the same Hilbert order the tree indexes. That is the whole trick, and it is why FlatGeobuf remains the right delivery format for feature-level reads even in a world that has otherwise standardised on columnar storage. This page sits under comparing GeoParquet vs FlatGeobuf performance and goes deep on the read path specifically.

#Quick Reference

#Two Structures, Two Very Different Read Paths

A GeoParquet reader is answering which rows match, and which columns do I need. It reads the footer to learn where each row group’s column chunks live and what their statistics say, skips the groups whose bounding boxes cannot overlap, and fetches the column chunks it needs from the survivors. The unit of work is the column chunk, and the payoff is that a query touching three of twenty columns transfers roughly three twentieths of the data.

A FlatGeobuf reader is answering which features are here. It fetches the header and the index, walks the R-tree locally, and reads the byte ranges holding the matching features. The unit of work is the feature, and the payoff is that the first feature is decodable the moment its bytes arrive — there is nothing to decompress and nothing to reassemble from separate columns.

The absence of internal compression is the deliberate part. Compressing features would mean a byte offset no longer identifies a feature — you would have to decompress a block to find anything inside it, which is exactly the indirection the format exists to avoid. FlatGeobuf pays roughly 60–80% more storage than a ZSTD-compressed GeoParquet of the same data, and buys the ability to read any window of it with no state.

#Reading a Window

# Requires: fiona>=1.10 (GDAL 3.8+) or pyogrio>=0.9, requests>=2.32  (Python 3.10+)
from __future__ import annotations

from dataclasses import dataclass

import pyogrio
import requests


@dataclass(frozen=True)
class WindowRead:
    features: int
    bytes_transferred: int
    requests_issued: int


def read_window(
    url: str,
    bbox: tuple[float, float, float, float],
    *,
    columns: tuple[str, ...] | None = None,
    timeout: float = 30.0,
) -> WindowRead:
    """Read features intersecting bbox from a remote FlatGeobuf.

    The driver walks the file's own R-tree and issues range requests; nothing
    is downloaded that the window does not intersect.
    """
    min_x, min_y, max_x, max_y = bbox
    if min_x >= max_x or min_y >= max_y:
        raise ValueError(f"degenerate bbox {bbox}")

    # A HEAD confirms the origin supports ranges before the driver commits.
    try:
        head = requests.head(url, timeout=timeout, allow_redirects=True)
        head.raise_for_status()
    except requests.RequestException as exc:
        raise RuntimeError(f"{url} is not reachable: {exc}") from exc
    if head.headers.get("Accept-Ranges", "").lower() != "bytes":
        raise RuntimeError(
            f"{url} does not advertise byte ranges — the whole file would download"
        )

    try:
        frame = pyogrio.read_dataframe(
            f"/vsicurl/{url}",
            bbox=bbox,
            columns=list(columns) if columns else None,
        )
    except Exception as exc:                      # driver raises a variety of types
        raise RuntimeError(f"windowed read of {url} failed: {exc}") from exc

    return WindowRead(
        features=len(frame),
        bytes_transferred=int(frame.memory_usage(deep=True).sum()),
        requests_issued=-1,                      # see the validation section
    )

#Validation

The number worth checking is requests issued, because it is the one that degrades silently when a file is written without Hilbert ordering. GDAL reports its range reads when the curl debug channel is on:

CPL_CURL_VERBOSE=YES CPL_DEBUG=ON \
  ogrinfo -al -so -spat 2.2 48.8 2.5 48.9 \
  /vsicurl/https://cdn.example.com/parcels.fgb 2>&1 | grep -c 'Range:'
# Healthy: single digits — header/index read plus a couple of feature spans.
# Unhealthy: dozens or hundreds — the file was written without spatial ordering,
# so matching features are scattered and every one costs its own request.

Expected ranges for a city-scale window against a national layer: 3–8 range requests, 2–20 MB transferred, and a first decoded feature within one round trip of the index read. If bytes transferred approaches the file size, the index is missing entirely and the driver has fallen back to a sequential scan.

#Edge Cases and Caveats

Files written without the index. A FlatGeobuf can legally be written unindexed, and several writers do so by default when the feature count is unknown up front. The file is valid, readers accept it, and every windowed query silently becomes a full sequential scan. Assert the index is present at publish time rather than discovering the absence through a slow map.

Attribute filters do not benefit from the index. The R-tree indexes geometry only. A query like “parcels in this window where use_class = ‘residential’” uses the index for the window and then evaluates the attribute predicate on every feature it read. If attribute selectivity is what matters and the spatial window is broad, a columnar format with column pruning will beat it comfortably.

Very large single files. The index sits between the header and the features, so its size grows with feature count and a multi-hundred-million-feature file has an index of tens of megabytes that every reader fetches before any feature. Past roughly 50 million features, splitting by region keeps the fixed read small, in the same way tile archives split past 20 GB.

#Frequently Asked Questions

How does FlatGeobuf answer a spatial query without a database?

The file carries a packed Hilbert R-tree immediately after its header, laid out so every node’s children sit at computable offsets. A client fetches the header, walks the tree over the bytes it already has, and learns the byte ranges of the features whose bounding boxes intersect the query window. Because features are stored in the same Hilbert order the tree indexes, those ranges are mostly contiguous, so a window query becomes two or three range reads rather than thousands.

When is FlatGeobuf a better choice than GeoParquet?

When the consumer wants whole features in a spatial window and wants the first one quickly. FlatGeobuf is row-oriented, so a feature’s geometry and all its attributes arrive together and can be rendered immediately. GeoParquet is columnar and wins decisively when a query touches a few columns across many rows, or aggregates. Delivery to a map favours the first; analysis favours the second.

Does FlatGeobuf compress its contents?

Not internally — the format stores features as uncompressed FlatBuffers so that a byte offset always identifies a feature and a reader can seek without decompressing anything. That is a deliberate trade: files are noticeably larger than a ZSTD-compressed GeoParquet, and in exchange any range of the file is directly usable. Transport compression at the HTTP layer is incompatible with range addressing, so the size is what it is.

#Related

• Comparing GeoParquet vs FlatGeobuf Performance — parent guide: the full format comparison and benchmark set
• How to Choose Between GeoParquet and FlatGeobuf — the decision rule, stated as a short checklist
• Space-Filling Curves for Spatial Partitioning — the ordering that makes the index’s answers contiguous
• PMTiles and Cloud-Native Tile Archives — the alternative when the consumer is a map rather than a feature client

← Back to Comparing GeoParquet vs FlatGeobuf Performance