pgokf

A PostgreSQL extension that turns Open Knowledge Format bundles into a queryable, transactional catalog - full-text, semantic, and hybrid search, a link graph, multi-tenant isolation, version history, and an audit trail, all inside PostgreSQL.

An OKF bundle is just a directory of UTF-8 Markdown "concept" documents with YAML frontmatter - runbooks, wikis, service catalogs, datasets. The bundle on disk stays the portable source of truth; pgokf materializes it into a PostgreSQL projection optimized for search and graph queries, and keeps them in sync incrementally.

• Extension & schema: pgokf · OKF conformance: v0.2 · PostgreSQL: 15–19
• Built with: Rust (edition 2024) + pgrx 0.19 · Safety: #![forbid(unsafe_code)], clippy-pedantic, cargo deny

---

Highlights

• 🔎 Search, four ways. Native PostgreSQL full-text ranking out of the box; optional BM25 (via Tiger Data pg_textsearch or ParadeDB pg_search) and semantic + hybrid RRF search (via pgvector) behind the same seam - none of them a hard dependency. Structured filters (type/tags/status/trust_tier), keyset pagination, faceted counts, and find_similar more-like-this.
• 🕸️ Link graph. Markdown cross-links and OKF Attested Computation references become resolved edges; concept_neighbors walks them (bounded, cycle-safe, BFS).
• 🏢 Multi-tenant. Opt-in row-level-security isolation keyed on a session tenant, backward-compatible (unset = see-all, or deny-by-default with require_tenant) - read and write confined.
• 🕰️ Version history. Opt-in point-in-time trail: concept_history and concept_as_of('… last Tuesday').
• 🧾 Audit & lifecycle. A durable sync log + per-sync change manifest, an exfiltration/access log, reversible retire/purge, and cross-bundle dedup.
• 📥 Two ingestion paths. From a filesystem path, or mountless - bytes streamed from an S3-compatible object store, with the extension performing zero network I/O.
• 🧰 Companion tools. Object-store ingestion, a reference embedder, an MCP server that exposes the catalog to AI agents, and a web UI with search, browsing, provenance, link graphs, an agent-plugin builder, and operations views - read-only until you give it a writer connection and a way of knowing who is asking, which turns on upload, edit and review.
• 📦 Operable. catalog_stats() / health() / search_index_status(), Parquet + source-file exports, pg_cron scheduled refresh, pg_dump-complete backups, PGXN / .deb / .rpm packaging, and multi-architecture Docker images (amd64 + arm64, so Apple Silicon too) that bundle pgvector, pg_cron, and pg_textsearch (on the PostgreSQL 17 and 18 images).

See the exact, versioned surface - every function, table, type, GUC, and role - in docs/sql-api.md and docs/api-stability.md.

Quick start

Registration runs in the PostgreSQL server process, so the bundle path must be absolute and server-reachable, and it requires membership in pgokf_writer. A sample bundle ships in examples/sample-bundle/.

CREATE EXTENSION pgokf;                       -- schema, tables, roles, functions
GRANT pgokf_writer TO myuser;                 -- reader < writer < admin

-- ingest a bundle (writer)
SELECT * FROM pgokf.register_bundle('/abs/path/to/examples/sample-bundle');

-- ranked full-text search, optionally filtered
SELECT concept_id, title, rank
FROM pgokf.concept_search('postgres failover', concept_type => 'runbook');

-- walk the resolved link graph
SELECT * FROM pgokf.concept_neighbors('runbooks/database-failover', 2);

-- browse the projection
SELECT id, title, type, tags FROM pgokf.concepts ORDER BY id;

Semantic / hybrid search (needs pgvector): supply embeddings with set_concept_embedding (or the pgokf-embed companion), build the index with rebuild_embedding_index(), then:

SELECT * FROM pgokf.concept_search_semantic( $query_vector );          -- nearest by cosine
SELECT * FROM pgokf.concept_search_hybrid('failover', $query_vector);  -- RRF fusion of lexical + vector

Optional capabilities

Everything works with stock PostgreSQL; these unlock more when installed, and degrade cleanly when absent:

Extension	Unlocks	Absent behavior
pgvector	concept_search_semantic, concept_search_hybrid, embeddings	semantic errors clearly; hybrid falls back to lexical
pg_textsearch (Tiger Data, PostgreSQL license, PG 17-18)	BM25 ranking (search_backend = bm25, the auto provider)	falls back to native FTS with a warning
pg_search (ParadeDB, AGPL-3.0)	BM25 ranking (bm25_provider = pg_search)	falls back to native FTS with a warning
pg_cron	schedule_refresh / unschedule_refresh / list_scheduled_refreshes	scheduling and listing raise a clear "install pg_cron" error

Companion tools

Standalone binaries (in crates/) that pair with the extension - credentials live in the companion, never in PostgreSQL, and each can connect over TLS. They ship together in the multi-architecture ghcr.io/logicocean/pgokf-companions:<version> image:

Tool	What it does
pgokf-ingest	Mountless ingestion: reads an S3/MinIO/SeaweedFS bucket and streams it into the catalog. --watch re-syncs on change.
pgokf-embed	Reference embedder: computes vectors via any OpenAI-compatible /v1/embeddings endpoint and stores them. --watch keeps up with new content.
pgokf-mcp	A Model Context Protocol server exposing concept_search / find_similar / concept_neighbors as agent tools, over stdio or over HTTP with bearer tokens and roles.
pgokf-web	The web UI and JSON API: search, browsing, concept pages, link graphs, the agent plugin builder, and operations - read-only until a writer connection and an identity mode turn on upload, edit and review.

Documentation

Full docs are published at https://logicocean.github.io/pgokf/. Key entry points:

Document	Covers
getting-started	Install, create the extension, grant a role, first queries
sql-api	Reference for every pgokf.* function, table, type, and GUC
architecture	Parser, sync engine, projection seams, search backends
search-guide	Ranking, filters, pagination, BM25, semantic + hybrid
multi-tenancy	Tenant isolation model, RLS, and its trust boundaries
version-history	Opt-in temporal history and point-in-time queries
deployment-topologies	Storage tiers, bucket-mount, mountless ingestion, Parquet
compose-deployment	The reference Docker Compose production stack (multi-arch images with pgvector, pg_cron, pg_textsearch; embedding daemon; backups)
operations · configuration	Day-2 ops, monitoring, upgrades; GUCs and policy keys
security	Roles, SECURITY DEFINER model, path containment, least privilege
okf-authoring	Authoring OKF v0.2 bundles (frontmatter, actors, reserved files)
api-stability	The public API contract, SemVer policy, deprecation

Runnable SQL is in examples/queries/; reusable OKF templates in templates/; authoring/catalog skills in skills/.

Building from source

cargo install cargo-pgrx --version 0.19.2 --locked
cargo pgrx init --pg18 $(which pg_config)                 # or your target major
cargo pgrx install --pg-config $(which pg_config) --features pg18

Select the major via the crate feature (pg15…pg19; default pg18). Run the gate:

cargo test -p pgokf --no-default-features --features pg18                     # unit + api-stability
RUST_TEST_THREADS=1 cargo pgrx test pg18 --no-default-features --features pg18 # in-database

Run the in-database suite single-threaded - see CONTRIBUTING.md for why, and for PGOKF_TEST_PRELOAD, which turns on the tests that need a preloaded BM25 provider.

Project status

Pre-1.0 (0.2.x). The enumerated SQL surface is treated as stable and every change ships an upgrade script verified upgrade == fresh, but per SemVer a 0.MINOR bump may still carry a breaking change (called out in CHANGELOG.md). Reaching 1.0.0 is a deliberate decision, not an automatic bump.

The extension builds from source on PostgreSQL 15–19 (pg15…pg19). Published binaries and Docker images cover 15–18; a 19 image builds once PGDG ships packages, and carries no BM25 provider until Tiger Data publishes a pg19 pg_textsearch.

License

Copyright (c) 2026 LogicOcean.

pgokf is dual-licensed: AGPL-3.0-only for all crates (LICENSE; every first-party source file carries an SPDX header), plus a commercial license for use the AGPL does not permit - embedding in a proprietary product, offering it as a managed service without releasing source, or an organizational no-AGPL policy. See LICENSING.md for the model and COMM-LICENSE.md for the commercial terms.

Security & contributing

Report vulnerabilities privately - see SECURITY.md. Contributions are welcome under a CLA (required by the dual-license model) - see CONTRIBUTING.md.