Anything-RAG

Graph intelligence middleware for existing RAG stacks. Plug into your current vector database pipeline, build a lightweight organizational memory layer in SQLite, and improve multi-hop retrieval without migrating infrastructure.

What it does

The problem

Most RAG systems retrieve by semantic similarity — "find chunks that sound like the question."

That works for simple lookups. It breaks when the answer depends on relationships spread across documents:

"Which customers migrated from Salesforce and then complained about billing?"

The migration might be in an account review. The complaint in a support ticket. Neither chunk alone looks like the full question — so vector search often returns the wrong chunks, or misses the connection entirely. The LLM then guesses.

Before vs after

Your data (two chunks):

Chunk	Text
ticket-42#1	Acme Corp complained about billing exports after migrating from Salesforce.
account-review#3	Acme Corp uses the reporting dashboard weekly.

Question: "Acme Corp customers who migrated from Salesforce and complained about billing"

	Without anythingRAG	With anythingRAG
How it searches	Embedding similarity only	Vector search + graph traversal
What it knows	"These chunks mention similar words"	Acme Corp → migrated_from → Salesforce, Acme Corp → complained_about → billing
Top result	Often account-review#3 (vaguely related)	ticket-42#1 (proves both relationships)
LLM outcome	May hallucinate or say "not found"	Gets the right evidence chunk

The ROI

You get	Without changing
Better multi-hop answers — connect facts across chunks	Your vector DB (Pinecone, pgvector, Chroma, etc.)
Less hallucination — retrieve evidence, not just similar text	Your ingest pipeline
Traceable reasoning — every relationship links back to a source chunk	Your LLM or LangChain stack
No migration — plug in middleware, keep everything else	Your infrastructure

Cost: one extra API call on ingest, one on query. Storage: a small SQLite graph (entities + relationships only — not your documents).

How it works (30 seconds)

anythingRAG sits on top of your existing RAG — it does not replace your vector database.

1. Ingest — send each chunk through anythingRAG; it extracts entities and relationships into SQLite, then you store the chunk in your vector DB as usual.
2. Query — run your normal vector search, pass results to anythingRAG; it boosts chunks that match graph constraints in the question.

Your vector DB stays the source of truth for text and embeddings. SQLite holds only derived knowledge: entities, relationships, evidence links, and ontology metadata.

Architecture

Customer App ──ingest──▶ anythingRAG Middleware ──▶ SQLite (graph projection)
       │                        │
       └──── chunk (unchanged) ─┴──▶ Customer Vector DB

Customer App ──query──▶ anythingRAG ──plan──▶ graph traversal
       │                     │
       └──── vector search ──┴──▶ merged ranked results

Quick start

1. Configure ontology

Copy and edit the example ontology:

cp ontology.example.yaml ontology.yaml

Define allowed entity types (Customer, Tool, Feature, …) and relationships (uses, migrated_from, complained_about, …).

2. Start services

Option 1 — One command (recommended for local dev)

./start.sh

Or:

./scripts/dev.sh

This starts both services together:

Service	URL
Extraction (Python)	http://127.0.0.1:8090
Middleware (Rust)	http://127.0.0.1:8787

Press Ctrl+C to stop both.

Optional — use GLiNER instead of the pattern fallback:

cd extraction-service
python3 -m venv .venv && source .venv/bin/activate
pip install gliner
cd .. && ./start.sh

The dev script automatically uses extraction-service/.venv when it exists.

Option 2 — Docker Compose

docker compose up --build

Option 3 — Two terminals (manual)

# Terminal 1: extraction sidecar
cd extraction-service && python3 app.py

# Terminal 2: middleware
cd middleware-service
export ANYTHINGRAG_ONTOLOGY_PATH=../ontology.example.yaml
export GLINER_SERVICE_URL=http://127.0.0.1:8090
cargo run --release

Middleware listens on http://127.0.0.1:8787.

3. Use the JavaScript SDK

import { AnythingRAG } from '@anythinggraph/anything-rag';

const rag = new AnythingRAG({
  baseUrl: 'http://127.0.0.1:8787',
  vectorSearch: async (query, topK) => myPineconeSearch(query, topK),
});

// Before writing to your vector DB
await rag.ingestChunk({
  chunkId: 'doc-1#chunk-3',
  text: 'Acme Corp complained about billing after migrating from Salesforce.',
  sourceDocId: 'doc-1',
});

// At query time
const results = await rag.search(
  'customers who migrated from Salesforce and complained about billing',
  { topK: 10 }
);

API endpoints

Endpoint	Method	Description
/health	GET	Service health
/ingest_chunk	POST	Extract entities/relationships from a chunk
/delete_chunk	POST	Remove chunk evidence from graph
/query_plan	POST	Build graph-aware retrieval plan
/search	POST	Merge graph plan with vector results
/ontology	GET	List approved entity/relationship types
/ontology/suggestions	GET	List pending ontology suggestions
/ontology/suggestions/:id/approve	POST	Approve a suggested type

Ingest example

curl -X POST http://127.0.0.1:8787/ingest_chunk \
  -H 'Content-Type: application/json' \
  -d '{
    "chunk_id": "ticket-42#1",
    "text": "Acme Corp complained about billing exports after migrating from Salesforce.",
    "source_doc_id": "ticket-42"
  }'

Search example

curl -X POST http://127.0.0.1:8787/search \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "customers who complained about billing",
    "top_k": 5,
    "vector_results": [
      {"chunk_id": "ticket-42#1", "score": 0.72}
    ]
  }'

GLiNER extraction

The Python extraction sidecar supports GLiNER when installed:

pip install gliner

Without GLiNER, a pattern-based fallback extractor is used automatically — good enough for local development and demos.

Environment variables

Variable	Default	Description
ANYTHINGRAG_LISTEN	0.0.0.0:8787	Middleware bind address
ANYTHINGRAG_DB_PATH	./data/anything-rag.sqlite	SQLite graph database path
ANYTHINGRAG_ONTOLOGY_PATH	./ontology.example.yaml	Ontology config file
GLINER_SERVICE_URL	http://127.0.0.1:8090	Extraction sidecar URL
ANYTHINGRAG_GRAPH_BOOST_WEIGHT	0.35	Score boost for graph-matched chunks

Project layout

anythingRAG/
├── middleware-service/   # Rust + Axum API
├── extraction-service/   # Python GLiNER sidecar
├── sdk/js/               # JavaScript SDK
├── ontology.example.yaml # Sample ontology
├── examples/             # Demo scripts
├── scripts/dev.sh        # Start both services locally
├── start.sh              # One-command local startup
└── docker-compose.yml

Ontology governance

When extraction discovers entity or relationship types not in your ontology, they are stored as pending suggestions. Approve them via API or SDK before they are used in future ingest operations.

License

Apache-2.0 — see LICENSE.