A monorepo of cutting-edge AI research, autonomous agents, and experimental LLM demos.
Welcome to the AI Forge Lab — a centralized hub for projects exploring the boundaries of LLM reasoning, autonomous workflows, and innovative RAG architectures. Each project lives in its own GitHub repository and is wired into this lab as a git submodule, so you can clone everything at once or pick a single project to dive into.
- Featured Projects
- Getting Started
- Working with Individual Submodules
- Research Themes
- Contributing
- License
Project names below match their upstream GitHub repository names. Click any project to open its dedicated repo.
| Project | Description | Key Tech |
|---|---|---|
openevolve-guide |
Framework for building self-evolving AI agents that improve their own code. | Python, LLMs, Process Isolation |
auto-research-guide |
Automated technical and scientific research workflows powered by AI. | LangChain, Research APIs |
finetuning-guide |
Comprehensive reference for fine-tuning LLMs (LoRA, QLoRA, DPO, RLHF). | PEFT, bitsandbytes, TRL |
mcp-chatbot |
A modular chatbot implementation using the Model Context Protocol (MCP). | LangChain, MCP, Groq |
podcast-automate |
End-to-end automation for podcast production, from script to speech. | Maya1, TTS, LLMs |
a2a-guide |
Action-to-Action guide: practical patterns for chaining agent actions. | Examples, Guides |
vectorless-llm-evals |
Evaluating RAG systems using PageIndex for hierarchical, vectorless retrieval. | PageIndex, LangSmith |
graph-llms |
Integrating Knowledge Graphs (Neo4j) with LLM reasoning for complex queries. | Neo4j, Cypher, Groq |
attention-to-gpt-scratch |
Build, deconstruct, and train decoder-only Generative Pre-trained Transformers. | PyTorch, tiktoken, GPT-2 |
agentic-deep-reader |
Multi-agent, multi-pass pipeline for extracting compliance evidence from long PDFs. | FastAPI, Groq, Datalab OCR, BM25 |
bm25-rlm-agent |
In-memory deep-research agent over a local markdown corpus using recursive decomposition. | FastAPI, LangChain, LangGraph, OpenAI |
llm-rpg-world-simulator |
Interactive RPG sandbox with Gemini world-transition engine, memory reflection, and emergent economy. | FastAPI, Gemini, NetworkX, Cytoscape.js |
mem-git-agent |
Self-contained MemGPT agent with tiered memory and versioned "memory git". | FastAPI, SQLite, NumPy, OpenAI, MarkItDown |
turboquant-personal-ai |
Folder-based personal AI with perfect memory via TurboVec quantization and hybrid retrieval. | FastAPI, SQLite, TurboVec, FAISS, OpenAI |
okf-knowledge-graph-wiki |
Ingests heterogeneous files, analyzes dependencies, and builds an AI-queryable OKF wiki. | Python 3.14, FastAPI, Gemini 3, SQLite, sqlite-vec, NetworkX |
omni-agent |
Agentic research harness executing parallel sub-agents with planning and self-correction. | Python 3.14, FastAPI, Gemini, SQLAlchemy, NetworkX, MarkItDown |
llm-distillation-guide |
A hands-on guide to LLM knowledge distillation: using Llama-3 (via Groq) to label synthetic data and fine-tuning a 1000x smaller RoBERTa student. | Llama-3, Groq, RoBERTa, PyTorch, Hugging Face |
rlm-due-diligence-agents |
Autonomous investment analyst leveraging LangChain Deep Agents & RLM-style dynamic workflows to run end-to-end technical, market, and financial due diligence with human-in-the-loop validation. | LangChain, Deep Agents, RLM, Python |
omni-video-agent |
End-to-end automated video generation: researches the web, writes a storyboard, generates clips (Veo 3.1) and music (Lyria 3), and compiles the final video. | LangChain Deep Agents, Gemini (Veo 3.1, Lyria 3, Nano Banana 2), crawl4ai, MarkItDown, FastAPI, SQLite, ffmpeg |
multidoc-agent-java |
Claude-Code-style multi-doc Q&A agent: planner + recursive research sub-agents over hybrid dense + BM25 + knowledge-graph retrieval, with cited answers. | Java 25, LangChain4j, Gemini, Javalin, SQLite, Lucene, MarkItDown |
dictate |
System-wide voice dictation and select-to-edit desktop app: press a hotkey, speak, and text is transcribed and pasted at the cursor, or used as an AI edit instruction if text is selected. | Tauri, Rust, TypeScript, AssemblyAI, Gemini |
office-ai |
Browser-based AI office suite: an agent builds spreadsheets, documents, and presentations live in an editable canvas next to the chat, with real .xlsx/.docx/.pptx import/export. |
Next.js, React, FastAPI, Univer, LiteLLM, Gemini |
Local paths: each repo is mounted in this monorepo at a submodule path. See the mapping in Working with Individual Submodules.
This repository uses Git Submodules to manage individual projects.
git clone --recursive https://github.com/subho004/ai-forge-lab.git
cd ai-forge-labgit submodule update --init --recursiveEach project resides in its own directory and has a dedicated README.md with installation and usage instructions.
Each project's display name = its GitHub repo name. The table below maps that repo (and URL) to the local submodule path inside this monorepo.
To initialize or update a single submodule after cloning:
# Example: agentic-deep-reader lives at ./long-document-context
git submodule update --init long-document-context
cd long-document-context
git pull origin main
cd ..Paths containing special characters (e.g.
okf+llm wiki) must be quoted:git submodule update --init "okf+llm wiki"
openevolve-guide — Self-evolving AI agents (openevolve-guide)
Framework for building self-evolving AI agents that improve their own code.
Key tech: Python, LLMs, Process Isolation.
auto-research-guide — Automated research workflows (auto-research-guide)
Automated technical and scientific research workflows powered by AI.
Key tech: LangChain, Research APIs.
finetuning-guide — LLM fine-tuning reference (finetuning-guide)
Comprehensive reference for fine-tuning LLMs (LoRA, QLoRA, DPO, RLHF).
Key tech: PEFT, bitsandbytes, TRL.
mcp-chatbot — Modular MCP chatbot (mcp-chatbot)
A modular chatbot implementation using the Model Context Protocol (MCP).
Key tech: LangChain, MCP, Groq.
podcast-automate — Podcast production automation (podcast-automate)
End-to-end automation for podcast production, from script to speech.
Key tech: Maya1, TTS, LLMs.
a2a-guide — Action-to-Action agent patterns (a2a-guide)
Action-to-Action guide: practical examples and patterns for chaining agent actions.
Key tech: Examples, Guides.
vectorless-llm-evals — Vectorless RAG evaluation (vectorless-llm-evals)
Evaluating RAG systems using PageIndex for hierarchical, vectorless retrieval.
Key tech: PageIndex, LangSmith.
graph-llms — Knowledge-graph reasoning (graph-llms)
Integrating Knowledge Graphs (Neo4j) with LLM reasoning for complex queries.
Key tech: Neo4j, Cypher, Groq.
attention-to-gpt-scratch — GPT from scratch (gpt-scratch)
Build, deconstruct, and train decoder-only Generative Pre-trained Transformers.
Key tech: PyTorch, tiktoken, GPT-2.
agentic-deep-reader — Long-context compliance pipeline (long-document-context)
Multi-agent, multi-pass pipeline for extracting and synthesizing compliance evidence from long PDFs.
Key tech: FastAPI, Groq, Datalab OCR, BM25.
bm25-rlm-agent — Recursive research agent (rlm-test)
In-memory deep-research agent over a local markdown corpus using recursive decomposition.
Key tech: FastAPI, LangChain, LangGraph, OpenAI.
llm-rpg-world-simulator — Generative RPG sandbox (world-simulator)
Interactive RPG sandbox with Gemini world-transition engine, memory reflection, and emergent economy.
- World Transition Engine: LLM parses player intent and NPC actions to output structured state diffs, processed via a validation choke point (
DiffApplier). - Emergent Quests: short supply triggers fetch quests; completing them pays rewards, resolves shortages, and spawns rumors.
- Cognitive Agent Memory: episodic memories are periodically reflected upon and consolidated into abstract insights.
- Social Cliques & Rumor Decay: community detection groups NPCs, while rumors decay in truth value as they propagate through the locations graph.
- Interactive Sandbox UI: visual Cytoscape.js map, command parser, detail logs, and save/load state.
Companion docs inside the submodule: docs/GAME_GUIDE.md, docs/MANUAL.md, docs/future_notes.md.
mem-git-agent — MemGPT tiered-memory agent (memgpt-test)
Self-contained MemGPT agent with tiered memory (persona, human, document catalog, recall, archival) and versioned "memory git".
- Tiered Memory Hierarchy: maps OS memory concepts to LLMs — editable core memory (
persona/human) always in context (RAM), a message FIFO queue, a document catalog, with full transcripts (recall) and embedded passages (archival) on disk. - Control Loop & Heartbeats: decouples thinking/acting from speaking via an inner monologue and automated heartbeats, chaining multiple search/edit calls before replying via
send_message. - Document Catalog Tier: always-in-context single-line metadata per document so the agent knows its whole library at a glance.
- Versioned "Memory Git": tracks all core-memory changes in a queryable, revertible database.
- Context Eviction & Summarization: manages token limits by prompting under memory pressure and recursively folding evicted history into a running summary.
- Hybrid Retrieval & Reranking: in-process NumPy similarity search fused with SQLite FTS5 BM25, then a cheap LLM rerank.
Companion docs: docs/notes.md.
turboquant-personal-ai — Ultra-low-bit vector memory (turboquant-test)
Folder-based personal AI assistant with perfect memory powered by TurboVec quantization and hybrid retrieval.
- TurboVec Vector Index: Rust implementation of Google Research's TurboQuant for 2-bit/4-bit quantization with AVX-512/NEON SIMD flat search — no index training or codebooks, high recall.
- Folder-Based Knowledge Management: uploads trigger MarkItDown parsing (or Whisper for audio), token-aware chunking, OpenAI embeddings, and multi-index updates.
- Follow-up Aware Conversation: rewrites multi-turn queries into standalone search queries.
- Persistent Conversation Memory: summarizes turns into a separate TurboVec memory index for cross-session recall.
- Cost & Token Tracker and Benchmark Dashboard (TurboVec vs. FAISS IndexPQ/IndexFlat).
Companion docs: docs/notes.md.
okf-knowledge-graph-wiki — Company Knowledge Builder (okf+llm wiki)
Ingests heterogeneous files, analyzes dependencies, and builds an AI-queryable OKF wiki.
- Universal Ingestion & Structure Analysis: heuristically analyzes project files (ZIP, PDF, DOCX, OpenAPI, SQL DDL) into a typed NetworkX dependency graph.
- Concept Page Extraction: structures key concepts with Gemini into Open Knowledge Format (OKF) markdown pages.
- Auto Wiki-Linking: resolves relative links between concepts, turning dangling references into stub pages.
- Hybrid Search & SSE Streaming:
sqlite-vecsemantic search + BM25 fused with RRF, streaming RAG chat and pipeline status over SSE.
Companion docs: docs/Company_Knowledge_Builder_OKF_Project_Idea.txt, docs/implementation_plan.md, docs/notes.md.
omni-agent — Multi-agent DAG research harness (harness-ultimate)
Agentic research harness executing parallel sub-agents with planning and self-correction.
- Multi-Agent DAG Planning: a NetworkX DAG executes tasks in parallel via specialized sub-agents.
- Self-Correction & Evaluation: critiques performance and dynamically reshapes/re-plans steps mid-flight.
- Recursive Sub-Agent Fan-Out: spawns nested sub-agents to delegate granular tasks in parallel.
- Grounding & Headless Browser Tools: ~16 real-world tools — Google Search grounding, browser-use/crawl4ai automation, sandboxed command execution, hybrid vector/BM25 retrieval.
Companion docs: docs/idea.md, docs/implementation-plan.md.
llm-distillation-guide — LLM knowledge distillation (LLM-distillation-guide)
Hands-on guide to LLM knowledge distillation: using Llama-3 (via Groq) to label synthetic data and fine-tuning a 1000x smaller RoBERTa student.
Key tech: Llama-3, Groq, RoBERTa, PyTorch, Hugging Face.
rlm-due-diligence-agents — Autonomous due diligence (deep-agents-due-diligence)
Autonomous investment analyst leveraging LangChain Deep Agents & RLM-style dynamic workflows to run end-to-end technical, market, and financial due diligence with human-in-the-loop validation.
Key tech: LangChain, Deep Agents, RLM, Python.
omni-video-agent — Automated video generation (agentic-video-generator)
End-to-end automated video generation from a single query: researches the web, storyboards scenes, generates video clips and music, and compiles the final video.
- Web Research & Synthesis: uses LangChain Deep Agents with crawl4ai (scraping) and MarkItDown (document parsing) to research a topic and build a structured knowledge wiki.
- Storyboard Generation: uses Gemini structured output to translate the research wiki into a shot-by-shot storyboard with Veo prompts, Lyria music briefs, and thumbnail prompts.
- Media Generation & Continuity: generates 30–60s video clips via Veo 3.1 (with native audio/dialogue and scene extensions) and composes a custom background music bed via Lyria 3.
- Assembly & UI: mixes and compiles video clips, narration, and background music using ffmpeg into a final
.mp4and creates a poster thumbnail with Nano Banana 2, served via a vanilla HTML/CSS/JS frontend.
Companion docs: docs/implementation-plan.md.
multidoc-agent-java — Agentic multi-doc Q&A (langchain4j-test)
Claude-Code-style planning agent over a pile of documents: decides what to search, spawns recursive research sub-agents, and answers with citations — hybrid dense + BM25 + knowledge-graph retrieval, single JVM, no Postgres/vector DB/Docker.
- Hybrid Retrieval Stack: Gemini dense embeddings + Apache Lucene BM25 + n-hop knowledge-graph expansion, fused with RRF and LLM reranking; adapts fusion weights by query type (exact-figure vs open-ended).
- Planner + Sub-Agents + Critic: Claude-Code-style plan → tool call → observe → revise loop; recursive
spawnResearchSubAgent, a critic that gates sufficiency, and an answer composer that cites every claim to a real chunk id. - Contextual Ingestion: MarkItDown sidecar parses PDF/Office/HTML/images; structure-aware chunking with Anthropic-style contextual enrichment before embedding; entity graph extracted per chunk into SQLite.
- Single-Binary Deploy: Java 25 + LangChain4j + Javalin — one
java -jar, oneGEMINI_API_KEY, conversations and SSE agent traces in a no-build-step browser UI.
Companion docs: docs/PLAN.md, docs/ARCHITECTURE.md, docs/NOTES.md.
dictate — System-wide voice dictation & select-to-edit (dictate)
Voice dictation and voice-driven editing, anywhere on the desktop. Press a global hotkey, speak, and the transcript is pasted at the cursor in any app on macOS or Windows — or, if text is selected first, the utterance is treated as an AI edit instruction applied in place.
- Selection-as-signal: no separate "command mode" or keyword triggers — whether text is selected at hotkey-press time decides plain dictation vs. AI edit.
- Real-time multilingual transcription via AssemblyAI, auto-detected with no configuration.
- Language-preserving edits: commands transform selected text without translating it, even across languages.
- Fast path for plain dictation: no LLM call at all unless there's a selection to act on.
- Native desktop app: Tauri (Rust) shell with a TypeScript/Vite frontend, works system-wide across editors, browsers, chat apps, and terminals.
Companion docs: docs/intent-state-management.md.
office-ai — AI office suite with a live editable canvas (office-ai)
Browser-based AI office suite — chat with an agent that reads, writes, and builds spreadsheets, documents, and presentations directly in a live, editable canvas next to the conversation, instead of round-tripping through generated files.
- Live canvas, not generated files: every agent tool call (setting a cell, writing a paragraph, adding a slide) repaints the same canvas a human would edit by hand; the user can take over at any point.
- Spreadsheets: multi-tab workbooks with real formulas and formatting via Univer, with
.xlsx/.csv/.odsimport/export. - Documents: a live word processor the agent writes and appends to, with
.docximport/export. - Presentations: a from-scratch PPTX engine (not a wrapper around an external converter) that edits real OOXML — add/reorder slides, move/resize shapes, recolor, edit text — while leaving untouched parts of the file byte-identical.
- Reference documents: attach existing spreadsheets, docs, or images to a chat for the agent to read from without disturbing the live canvas.
- Architecture: Next.js/React frontend, a thin FastAPI backend relaying tool calls over WebSocket, and LiteLLM fronting Gemini (or any provider) — the backend never touches document bytes directly.
Companion docs: docs/ (architecture notes, screenshots).
LLMs iteratively improving their own logic and tools through self-correction and performance-based evolution — see openevolve-guide.
Advanced alignment strategies including SFT, LoRA/QLoRA, preference optimization (DPO), and knowledge distillation to smaller student models — see finetuning-guide and llm-distillation-guide.
Moving beyond flat vector embeddings toward structured knowledge graphs for accurate, traceable RAG — see graph-llms and okf-knowledge-graph-wiki.
Streamlining the path from raw ideas to polished media like podcasts and videos — see podcast-automate and omni-video-agent.
Multi-agent pipelines synthesizing structured evidence from massive (1000+ page) corpora via iterative refinement, BM25 dependency resolution, and LangGraph self-correction — see agentic-deep-reader, bm25-rlm-agent, and multidoc-agent-java.
LLM-driven role-playing simulations with structured intent parsing, episodic memory reflection, emergent trade economies, and rumor propagation — see llm-rpg-world-simulator.
LLMs as operating systems: tiered memory hierarchies (RAM vs. disk), tool-driven memory paging, context eviction/summarization, and versioned memory histories — see mem-git-agent.
Ultra-low-bit quantization à la Google Research's TurboQuant (1536-dim → 2–4 bits, no training/codebooks) benchmarked against FAISS — see turboquant-personal-ai.
Decomposing complex queries into parallelized DAG execution plans or dynamic workflows with self-reflection, browser automation, sandboxed runtimes, hybrid retrieval, and human-in-the-loop verification — see omni-agent and rlm-due-diligence-agents.
Bringing agentic LLMs directly into everyday desktop and office workflows — system-wide voice dictation with select-to-edit, and a live, editable canvas for spreadsheets, documents, and presentations — see dictate and office-ai.
Contributions are welcome across all sub-projects! Since each project is an independent repository:
- Open issues and pull requests on the individual project's GitHub repo (linked in the tables above).
- Refer to each project directory's own
README.mdfor project-specific contribution guidelines. - For monorepo-level changes (this README, submodule wiring), open a PR against
ai-forge-lab.
This lab is released under the MIT License — see LICENSE for details. Individual submodules may carry their own licenses; check each repository.
Created with ❤️ by subho004