#Preserving QGIS Metadata in FlatGeobuf

To preserve QGIS metadata in FlatGeobuf, explicitly extract layer-level metadata from the GeoPackage source, serialize it as a validated JSON string, and inject it into the FlatGeobuf binary header using GDAL’s METADATA layer creation option. FlatGeobuf does not parse .qgz project files or carry QGIS symbology, relations, or custom XML blocks automatically. The format supports only a single, layer-scoped JSON field embedded in the file header. Treat metadata as an explicit serialization target — not an implicit export side effect — and use GDAL ≥ 3.6.0 with pyogrio ≥ 0.6.0 for reliable injection. This page is a companion to Preserving Metadata During GeoParquet Conversion, which covers the GeoParquet-specific approach using Parquet key-value footer pairs.

#Quick-reference: what survives the conversion

Primary use case: automated headless pipelines exporting to cloud storage where web GIS clients need layer-level provenance without a companion sidecar file.

#Why only layer-scoped JSON survives

The FlatGeobuf binary format is designed for streaming and low-latency HTTP range requests — its header is read once at stream initialization and must be small and fast to parse. To support this design, the specification provides exactly one metadata slot: a metadata string field in the layer header, containing arbitrary UTF-8 text. In practice GDAL’s driver treats this as a JSON object.

QGIS distributes its configuration across three distinct storage layers: project-level canvas state and relations live in .qgz (a zipped XML archive); layer properties live in GeoPackage metadata tables (gpkg_metadata and gpkg_metadata_reference); and feature attributes sit in the ordinary attribute table. When the GDAL FlatGeobuf driver converts a GeoPackage it copies the feature table faithfully but strips all QGIS XML tags and drops project-level context. Only key-value pairs you explicitly map survive.

The same architectural constraint applies when working with GeoParquet conversion pipelines — though GeoParquet stores metadata as Parquet key-value footer pairs rather than a single header string, making it slightly more flexible for multi-key provenance records. For data conversion and migration pipelines that target both formats, design a common metadata extraction step and then route serialization per format. Where schema mapping for legacy-to-modern formats is also in scope, handle CRS normalization and type coercion before the metadata extraction step to keep provenance consistent across output files.

#Production implementation

The following workflow queries gpkg_metadata via SQLite, falls back to ogrinfo -json for GeoPackages that pre-date the QGIS metadata table schema, validates the payload, and writes to FlatGeobuf with GDAL’s METADATA creation option. Requires GDAL ≥ 3.6.0, pyogrio ≥ 0.6.0, geopandas ≥ 0.14.0.

# pyogrio >= 0.6.0, geopandas >= 0.14.0, GDAL >= 3.6.0
import json
import sqlite3
import subprocess
from pathlib import Path
from typing import Any, Optional

import geopandas as gpd
import pyogrio

_METADATA_SIZE_LIMIT = 64 * 1024  # 64 KB — GDAL parser reliability threshold


def extract_qgis_layer_metadata(gpkg_path: str, layer_name: str) -> dict[str, Any]:
    """Extract QGIS layer metadata from GeoPackage metadata tables.

    Queries gpkg_metadata via SQLite. Falls back to ogrinfo -json
    if the metadata tables are absent (older GeoPackage schemas).
    """
    db_path = Path(gpkg_path)
    if not db_path.exists():
        raise FileNotFoundError(f"GeoPackage not found: {gpkg_path}")

    metadata: dict[str, Any] = {}

    try:
        with sqlite3.connect(str(db_path)) as conn:
            cursor = conn.cursor()
            cursor.execute(
                """
                SELECT m.metadata
                FROM gpkg_metadata m
                JOIN gpkg_metadata_reference r ON m.id = r.md_file_id
                WHERE r.table_name = ?
                """,
                (layer_name,),
            )
            for (raw_meta,) in cursor.fetchall():
                if not raw_meta:
                    continue
                try:
                    parsed = json.loads(raw_meta)
                    if isinstance(parsed, dict):
                        metadata.update(parsed)
                    else:
                        # Wrap non-dict JSON values so the outer payload stays a dict
                        metadata["qgis_metadata_value"] = parsed
                except json.JSONDecodeError:
                    metadata["qgis_raw_metadata"] = raw_meta

    except sqlite3.OperationalError:
        # gpkg_metadata tables absent — fall back to ogrinfo
        cmd = ["ogrinfo", "-json", "-so", gpkg_path, layer_name]
        result = subprocess.run(cmd, capture_output=True, text=True, check=True)
        ogr_json = json.loads(result.stdout)
        for layer in ogr_json.get("layers", []):
            metadata.update(layer.get("metadata_fields", {}))
            if "description" in layer:
                metadata["qgis_description"] = layer["description"]

    return metadata


def write_flatgeobuf_with_metadata(
    input_gpkg: str,
    output_fgb: str,
    layer_name: str,
    extra_metadata: Optional[dict[str, Any]] = None,
) -> None:
    """Write a GeoPackage layer to FlatGeobuf with injected header metadata.

    Args:
        input_gpkg: Path to source GeoPackage.
        output_fgb: Path for the output .fgb file.
        layer_name: Name of the layer to convert.
        extra_metadata: Optional dict merged into the extracted metadata
                        (useful for adding pipeline provenance keys).
    """
    metadata = extract_qgis_layer_metadata(input_gpkg, layer_name)
    if extra_metadata:
        metadata.update(extra_metadata)

    # Validate JSON serializability and enforce size limit
    try:
        meta_json = json.dumps(metadata, ensure_ascii=False)
        json.loads(meta_json)  # Round-trip validation
    except (TypeError, ValueError) as exc:
        raise ValueError(f"Metadata payload is not JSON-serializable: {exc}") from exc

    if len(meta_json.encode("utf-8")) > _METADATA_SIZE_LIMIT:
        raise ValueError(
            f"Metadata payload exceeds 64 KB ({len(meta_json)} bytes). "
            "Offload large records to an external STAC catalog."
        )

    gdf = gpd.read_file(input_gpkg, layer=layer_name, engine="pyogrio")

    # pyogrio >= 0.7: layer_creation_options
    # pyogrio 0.6.x: dataset_creation_options
    # Check installed version and use the appropriate parameter.
    pyogrio_version = tuple(int(x) for x in pyogrio.__version__.split(".")[:2])
    creation_kwarg = (
        "layer_creation_options" if pyogrio_version >= (0, 7) else "dataset_creation_options"
    )

    pyogrio.write_dataframe(
        gdf,
        output_fgb,
        driver="FlatGeobuf",
        **{creation_kwarg: {"METADATA": meta_json}},
    )
    print(f"Wrote {output_fgb} with {len(metadata)} metadata keys ({len(meta_json)} bytes).")


if __name__ == "__main__":
    write_flatgeobuf_with_metadata(
        input_gpkg="data/source.gpkg",
        output_fgb="data/output.fgb",
        layer_name="administrative_boundaries",
        extra_metadata={"pipeline_version": "1.4.0", "source_crs": "EPSG:4326"},
    )

#Validation and verification

After writing, confirm the metadata round-trips correctly using ogrinfo:

ogrinfo -json -so data/output.fgb

Look for the metadata key in the layers[0] object. The value should be the exact JSON string you injected. An absent or empty metadata key indicates a GDAL version below 3.6.0 or a pyogrio parameter name mismatch.

For automated CI/CD validation, integrate a jsonschema check that enforces required provenance keys before triggering any export. This guard belongs in the same pipeline stage as the batch conversion orchestration so that invalid payloads fail fast rather than producing silently broken files.

# jsonschema >= 4.0.0
import jsonschema

METADATA_SCHEMA = {
    "type": "object",
    "required": ["title", "description", "source", "last_updated"],
    "properties": {
        "title":        {"type": "string"},
        "description":  {"type": "string"},
        "source":       {"type": "string"},
        "last_updated": {"type": "string", "format": "date"},
    },
}

def validate_metadata(metadata: dict) -> None:
    """Raise jsonschema.ValidationError if required provenance keys are missing."""
    jsonschema.validate(instance=metadata, schema=METADATA_SCHEMA)

Expected output for a valid payload: jsonschema.validate returns None. An invalid or incomplete payload raises jsonschema.ValidationError before the GDAL write is attempted, preventing silent provenance loss.

#Edge cases and caveats

Payload above 64 KB from rich QGIS descriptions. Some QGIS layers accumulate extensive metadata through repeated edits — particularly layers that store original OGC XML metadata in gpkg_metadata. Serialized to JSON, these can exceed 64 KB. In that case, extract the high-signal keys (title, abstract, source, crs, last_updated) into a trimmed dict, and write the full record to an external STAC item or cloud storage object tag. The pipeline code above enforces this limit and raises a descriptive error rather than silently truncating.

Mixed CRS sources in multi-layer GeoPackages. When batch-converting all layers from a single GeoPackage, each layer’s CRS may differ. Always include source_crs and target_crs in the injected metadata, and reproject to a canonical CRS (typically EPSG:4326 for web delivery) before writing. Missing CRS metadata is a common failure mode in downstream null-value handling for spatial schema migration — the receiving schema may reject features where CRS fields are absent.

QGIS project-level settings cannot be embedded. Relations, joins, and virtual field expressions defined at the project level live in the .qgz XML and have no equivalent in the FlatGeobuf header. If downstream consumers need project-level context, write a companion JSON sidecar (e.g. output.fgb.meta.json) and reference it from the embedded header via a sidecar_url key. For web delivery, a STAC item describing the layer is the most interoperable approach. This is a fundamental architectural constraint of the format, not a GDAL limitation.

#FAQ

#Related

• Preserving Metadata During GeoParquet Conversion — column-level metadata injection into Parquet key-value footers, contrasted with FlatGeobuf’s single header string
• GeoParquet vs FlatGeobuf Performance Comparison — when to choose each format for cloud delivery workloads
• Schema Mapping for Legacy-to-Modern Formats — handling type coercion and null semantics when migrating GeoPackage schemas
• Building Batch Conversion Pipelines with Python — orchestrating multi-layer exports at scale
• Null Value Handling in Spatial Schema Mapping — CRS and attribute nullability edge cases that affect downstream metadata integrity

← Back to Preserving Metadata During GeoParquet Conversion