highwayscene-proto

highwayscene-proto is the reference Python reader and Protobuf schema for HighwayScene, a multi-sensor static roadside LiDAR dataset. It provides generated message classes, memory-efficient record iteration, random frame access, and typed NumPy views of point-cloud channels.

HighwayScene accompanies the paper “Beam-Wise Statistical Background Subtraction for Static Roadside LiDAR: A Cross-Sensor Benchmark Study.” See the project page for the paper, code, dataset, and benchmark resources.

Installation

Install the reader from PyPI:

python -m pip install highwayscene-proto

For development from a checkout:

python -m pip install -e '.[dev]'

Python 3.10 or newer is required.

Quick start

Stream frames from a record without loading the complete file:

from highwayscene import LaserName, decode_pointcloud, iter_frames

for frame in iter_frames("HighwayScene/test/record.pb"):
    print(frame.frame_id, frame.frame_timestamp_ns)
    for scan in frame.lidars:
        sensor = LaserName(scan.laser_name).name
        points = decode_pointcloud(scan.pointcloud)
        print(sensor, points.xyz.shape, points.channel_id)

Use RecordFile when you need indexed access:

from highwayscene import RecordFile, decode_pointcloud

record = RecordFile("HighwayScene/test/record.pb")
frame = record[12]
xyz = decode_pointcloud(frame.lidars[0].pointcloud).xyz

The existing generated-module imports remain available:

from highwayscene.frame import Frame
from highwayscene.sensors import LaserName, LidarScan

Imports through the former a42 package name remain available as compatibility aliases. New code should use the highwayscene namespace.

Inspect a file or a complete split from the command line:

highwayscene-inspect HighwayScene/test --limit 3
highwayscene-inspect HighwayScene/test/record.pb --json

Dataset download and layout

Download HighwayScene with Git LFS or huggingface_hub:

python -c "from huggingface_hub import snapshot_download; snapshot_download(repo_id='iis-esslingen/HighwayScene', repo_type='dataset', local_dir='HighwayScene')"

The point-cloud records are organized by benchmark split:

HighwayScene/
├── metadata/
│   ├── ground_truth.yaml
│   ├── manifest.json
│   └── SHA256SUMS
├── train/*.pb
├── val/*.pb
└── test/*.pb

The benchmark ground-truth definitions are stored in metadata/ground_truth.yaml and consumed by the roadside-lidar-background-subtraction repository. HighwayScene does not contain separate generated point-wise label files. The optional object_list field in the Protobuf schema is independent of these benchmark definitions.

Record format

Each .pb file is a sequence of messages. Every message consists of a four-byte unsigned little-endian payload length followed by one serialized highwayscene.frame.Frame message:

[uint32 payload bytes][serialized Frame][uint32 payload bytes][serialized Frame]...

The schema is available under proto/highwayscene. Cartesian coordinates are interleaved little-endian float32 values. Populated auxiliary channels contain one value per Cartesian point:

Field	NumPy dtype	Availability
cartesian	<f4, shaped (N, 3)	all sensors
intensity	<u2	sensor-dependent
ambient	<u2	sensor-dependent
velocity	<f4	Aeva Aeries II
reflectivity	<u2	sensor-dependent
timestamp_offset	<u8	sensor-dependent
channel_id	<u2	sensors with fixed beam IDs
horizontal_id	<u2	Blickfeld Qb2 scan positions

decode_pointcloud() validates byte alignment and channel lengths. Its default arrays are zero-copy, read-only views. Pass copy=True when writable arrays are required.

Sensor identifiers

The HighwayScene classification sensors use these enum values:

Sensor	LaserName
Blickfeld Qb2	CLASS_QB2
Ouster OS0	CLASS_OS0
Aeva Aeries II	CLASS_AERIES_II

The remaining enum values are retained for wire compatibility with the HighwayScene record format.

Schema generation

Generated betterproto bindings are committed so users do not need a Protobuf compiler. To regenerate them after an intentional schema change:

python -m pip install -e '.[codegen]'
./scripts/generate_bindings.sh

Never renumber or reuse an existing Protobuf field. CI verifies that committed bindings match the schemas.

Licensing

This reader and its Protobuf schemas are released under the MIT License. The HighwayScene dataset is distributed separately under CC BY-NC-SA 4.0; using this package does not change the dataset license.