GlamDB is an open-source columnar database written in C++23. It stores typed records in Parquet files, manages snapshots with Apache Iceberg, and uses SQLite for its catalog and write outcomes. Use it as an embedded C++ library or through the HTTP server, browser workbench, command-line client, and Python SDK.
GlamDB is a software project developed for a hackathon at ITMO University and is not intended for production use. Presentation about the project
GlamDB is in early development. The engine supports append and explicit key-value collections on local disk, with one owning process, one serialized writer per store, and concurrent snapshot readers. Power-loss durability qualification is still pending.
- Typed collections. Fixed schemas with strings, booleans, int64, float64, UTC microsecond timestamps, exact decimal128 amounts and calendar dates, and nullable fields. See scalar types and compatibility. Appends preserve every row, including duplicate keys. Explicit KV collections provide full-row upserts, safe keyed deletes, fixed-snapshot batched lookups, bounded streaming bulk upsert and locality-preserving compaction.
- Snapshot reads. Query the latest committed view or a retained historical snapshot. Select columns, limit results, and combine comparisons, membership, and NULL predicates with AND, OR, and NOT; scans prune candidate files and Parquet row groups conservatively and read only needed column ranges.
- Explicit write outcomes. Persistent operation IDs support exact retries and outcome lookup after restart. Writes distinguish committed, definitely not committed, and unknown outcomes.
- Open columnar formats. Arrow batches in memory, immutable Parquet data files, and Iceberg v2 tables with retained snapshot history. Collection and per-column Parquet compression, levels, encodings and dictionary policies are configurable through the SDK and workbench.
- Data inspection. Browse schemas, snapshot history, statistics, files, Parquet row groups, scan plans, and execution profiles.
- Native compute contract. Negotiate versions 1.0–1.5, operation support by append/KV mode and effective limits through the installed C++ API. See the storage contract. Contract 1.1 adds bounded native read views for repeated scans across a coherent per-store snapshot vector. Contract 1.2 adds bounded, repeatable parallel scan splits. Contract 1.3 adds scan pushdown negotiation with explicit exact/necessary-condition/residual filters and safe global limits. Contract 1.4 adds durable write sessions for one-pass append/KV output through a separate bounded staging journal and the existing Engine publisher. Contract 1.5 adds the snapshot archive (hot window, archived reads, maintenance), verified-checkpoint open and explicit full verification. Remote sessions and read views, stable C ABI and Flight remain unsupported.
- Connected tools. Manage multiple databases through one authenticated server. Read and write through the UI, CLI, or Python SDK; retrieve scan results as JSON or Arrow IPC with fixed-snapshot pagination.
Queries use a typed predicate API; there is no SQL query interface. Collections are unpartitioned with immutable append or KV mode. Schema evolution, general garbage collection, object storage and replication remain outside the supported profile. The embedded C++ API supports explicit snapshot retention maintenance for append collections; history is retained forever unless a caller opts in. KV collections preserve all history and refuse snapshot expiration.
Open the public demo — anyone can try it.
The organization workbench uses Keycloak
sign-in, OAuth2 Proxy sessions and Cerbos policies for isolated database access.
See the deployment and operating guide
for HTTPS hosting, native account management and database ownership workflows.
The combined-login guide also documents
the explicit temporary hello/world setup for an existing demonstration account.
The static Pages demo contains disposable sample data and remains a separate
publication; it does not connect to protected database servers.
Clone the repository and start a local server:
git clone --recurse-submodules https://github.com/glamourdb/glamdb.git
cd glamdb
export GLAMDB_TOKEN="$(openssl rand -hex 32)"
./scripts/glamdb-local.sh up
./scripts/glamdb-local.sh statusThe helper builds the server and its pinned dependencies, then starts it in the background. On Linux, it uses the native toolchain described under Build from source; on macOS, it uses a running Docker daemon. Allow 30–60 minutes for the first dependency build.
Open http://127.0.0.1:8080 and enter the value of GLAMDB_TOKEN.
The server starts with no databases. Initialized managed databases can be
explicitly removed through the SDK or glamdb-cli drop-database using their
recorded store UUID; glamdb-cli bench tpch clean --state RUN_STATE supports
dedicated benchmark cleanup. See the deletion and retry contract.
Create a database, create a collection with
its schema, then append and query rows in the workbench. The same UI provides
snapshot selection, storage inspection, and scan explain views.
Data survives server restarts. The default location is
~/.local/share/glamdb/data on Linux, or the glamdb-data Docker volume.
Set GLAMDB_DATA to choose another directory or volume and GLAMDB_PORT to
change the port. Writes require free space of at least the larger of 1 GiB or
5% of the data filesystem, plus a 512 MiB reserve.
./scripts/glamdb-local.sh logs -f # Follow server logs
./scripts/glamdb-local.sh down # Stop the server and keep its dataKeep the token for subsequent starts, or omit GLAMDB_TOKEN and let the helper
generate and reuse a token file; status shows its location. The service token
grants access to all databases. Remote connections use HTTPS through a TLS proxy;
see the server guide for deployment and API details.
The examples below use the local server and token from the quick start. Run commands from the repository root.
glamdb-cli is a standalone HTTP/HTTPS client. The local helper runs it with the
server's address and token. Create a database, define a collection, append two
rows, and read them back:
./scripts/glamdb-local.sh cli request --path /databases --body - <<'JSON'
{"name":"cli-example","database_id":"11111111-1111-4111-8111-111111111111"}
JSON
COLLECTION=/databases/11111111-1111-4111-8111-111111111111/collections
./scripts/glamdb-local.sh cli request --path "$COLLECTION" --body - <<'JSON'
{
"collection_id": "22222222-2222-4222-8222-222222222222",
"name": "events",
"schema": {"fields": [
{"name": "key", "type": "string", "nullable": false},
{"name": "value", "type": "int64", "nullable": true}
]}
}
JSON
COLLECTION=$COLLECTION/22222222-2222-4222-8222-222222222222
./scripts/glamdb-local.sh cli request --path "$COLLECTION/append" --body - <<'JSON'
{
"operation_id": "33333333-3333-4333-8333-333333333333",
"rows": [{"key": "a", "value": "1"}, {"key": "b", "value": "2"}]
}
JSON
printf '%s' '{}' | ./scripts/glamdb-local.sh cli request \
--path "$COLLECTION/scan" --body -Int64 values use decimal strings in JSON to preserve precision. The request
command exposes every HTTP route; --body implies POST. Scan responses include
next_cursor when another page is available. For scripts that consume complete
results, the Python SDK handles pagination automatically.
Named commands also cover create-collection, collections, resolve-collection,
collection, schema, append, scan, operation, snapshots, files, and
file. Use --url and --database-id with glamdb-cli. The optional
glamdb-admin offers the same commands offline with --root and --store-id,
or remotely with --url. Named scans complete HTTP pages within a 10,000-row,
16 MiB result bound. See the CLI guide for commands, input
formats and the distinct failure/unresolved exit codes.
Databases default to eight collections and accept a resource-backed opt-in
limit up to 64. Use create-store --max-collections 64 or the managed SDK/HTTP
limits object; managed limits survive restarts. Collection inventories support
fixed-view pages. See collection profiles
for configuration and inventory semantics.
Install the SDK from the checkout with Python 3.10 or later:
python3 -m venv .venv
. .venv/bin/activate
python -m pip install ./sdk/pythonimport os
from uuid import UUID
from glamdb_sdk import Client, Field, Schema, col
url = "http://127.0.0.1:8080"
token = os.environ["GLAMDB_TOKEN"]
schema = Schema([Field("key", "string", False), Field("value", "int64", True)])
collection_id = UUID("44444444-4444-4444-8444-444444444444")
operation_id = UUID("55555555-5555-4555-8555-555555555555")
with Client(url, token=token) as admin:
database = admin.create_database("python-example")
with Client(url, token=token, database_id=database["database_id"]) as client:
created = client.create_collection("events", schema, collection_id=collection_id)
if created.outcome != "committed":
raise RuntimeError(created.raw)
request = client.prepare_append(
collection_id,
[{"key": "a", "value": 1}, {"key": "b", "value": 2}],
operation_id=operation_id,
schema=schema,
)
outcome = client.append(request)
if outcome.outcome != "committed":
raise RuntimeError(outcome.raw)
result = client.scan_json(collection_id, predicate=col("value", "int64").ge(2))
print(result.rows) # [{"key": "b", "value": 2}]
with client.scan_batches(collection_id) as batches:
for batch in batches:
print(batch.to_pylist())These fixed UUIDs make the example repeatable. In an application, generate and save a collection ID and an operation ID before submitting each new create or append. Preserve the original IDs and payload if a response is lost; look up the outcome before deciding to retry. The SDK never retries writes implicitly. See the SDK guide for historical reads, metadata, streaming, and recovery handling.
Streaming bulk append accepts replayable batches as one operation and snapshot through C++, HTTP, Python and the CLI. Ordinary append keeps its separate 10,000-row/16 MiB limits. Bulk upsert and append-to-KV migration are embedded C++ operations.
The optional workload package provides complete TPC-H and ClickBench append/KV lifecycles, exact verification, saved-request recovery and benchmark cleanup. Scans use the typed API; reductions run in the client, including bounded DuckDB staging for ClickBench. These are not native SQL or official benchmark scores.
The installed glamdb::core CMake target provides CreateStore, OpenExisting,
Engine::CreateCollection, Engine::AppendDurable, and Engine::Scan through
<storage/engine.h>. Input and output use Arrow batches. The embedded library
and HTTP server share the same engine; an embedded owner must have exclusive
ownership of its store directory.
Native builds require Linux, CMake 3.28+, Ninja, Python 3.10+, OpenSSL and zlib
development headers, and a C++23 compiler with std::expected support: GCC 14+
or Clang 18+ with a matching standard library. Build dependencies and GlamDB
with the same compiler and C++ ABI.
git submodule update --init --recursive
cmake --preset deps-local
cmake --build --preset deps-local --parallel 2
cmake --preset local
cmake --build --preset local
ctest --preset localTo run the resulting server directly, with GLAMDB_TOKEN set:
./build/local/apps/server/glamdb --data-dir "$PWD/build/data" --ui-root "$PWD/ui"For a smaller shared memory profile, --http-readers 1 --http-readers-per-database 1 queues heavy metadata reads without restricting
control or publishing calls. Defaults remain four readers globally and two per
database; see the HTTP service profile
for the finite memory settings and admission limits.
The build also produces build/local/apps/cli/glamdb-cli. Use the release
or asan-ubsan preset instead of local for release or sanitizer builds.
cmake --install build/local installs binaries, UI assets, public headers, and
the CMake package into build/install/local.
For the full native test suite in Docker, including on macOS:
./scripts/build-deps-in-docker.sh
./scripts/test-storage-in-docker.sh asan-ubsanSee the build guide for toolchain setup, standalone client builds, and embedding the installed library. For four-state rules and native Telegram setup, see the alerting guide.
For metrics, Grafana dashboards and deployment steps, see the monitoring guide.
For a single Linux server, the complete observability quickstart provides one command for Prometheus, Grafana, Loki, Alloy, Alertmanager and host/container collectors. Real application endpoints and a dedicated Loki filesystem are inputs; repeated deployment preserves owned data and credentials.
New databases default to byte-bound retained-history budgets (the former 4,096-operation/snapshot horizon is gone; persisted databases keep their limits). The linked SF=10 recipe uses ordinary appends and preserves permanent outcomes, duplicate rows, historical views and exact retries.
The workbench development environment requires Python 3.13, Node.js 22+, and OpenSSL. It includes the SDK, Python tests, Node tests, and Playwright browser tests:
bash scripts/bootstrap-workbench.sh
npm --prefix ui exec -- playwright install chromium
bash scripts/check-workbench.shOn Linux, add --with-deps to the Playwright install command to install browser
system dependencies. Tests use localhost sockets and disposable stores.
See the documentation index for API references and operator guides.
Bug reports, documentation improvements, tests, and code contributions are welcome. Use issues to report a reproducible problem or discuss a proposed feature.
-
Fork
glamourdb/glamdband create a branch in your fork. -
Make a focused change, add tests for behavior changes, and update affected docs.
-
Run the relevant tests. For C++ changes, also run the quality checks with LLVM 18
clang-formatandclang-tidy:cmake --build --preset local --target format-check tidy-check quality-negative-check -
Open a pull request against
glamourdb/glamdb:main, describing the change and how you tested it. Storage changes should address recovery and compatibility.
Pull requests require passing checks and approval from another maintainer. See CONTRIBUTING.md for review and merge requirements. Report security vulnerabilities privately through the security policy.
GlamDB is licensed under the Apache License 2.0.
