A tutorial-scale, fully local RAG-based chatbot built with Gemma 3 via Ollama + Bright Data SERP API + Chroma. This researches topics comprehensively and then answers questions using retrieval-augmented generation. The system operates in two distinct phases: Topic Ingestion and Chat Mode.
- Two-Phase Operation: Research topics first, then engage in Q&A
- Comprehensive Topic Research: Uses 20+ search templates for thorough coverage via Bright Data SERP
- Local LLM Integration: Powered by Ollama with Gemma models
- Vector Database Storage: ChromaDB for efficient similarity search
- Intelligent Caching: Reduces API calls and improves performance
- Topic Management: Switch between different research topics
- Conversation History: Maintains context during Q&A sessions
- The models need about 4.5-5 GB of free space
- Gemma 3 can run on a single consumer-tier dGPU
- Gemma 3 4B IT QAT and ChromaDB running together should use about 6 GB VRAM at peak
- Node.js
- Ollama installed and running locally
- ChromaDB (automatically handled by the chromadb package) + Docker (get this yourself)
- Bright Data SERP API credentials
The system will automatically attempt to pull these models if missing:
gemma3:4b-it-qat(text generation)nomic-embed-text:latest(embeddings)
-
Clone or navigate to the project directory:
cd rag-chatbot -
Install dependencies:
npm install
-
Set up environment variables:
cp env.example .env
Edit
.envwith your Bright Data credentials:BRIGHT_DATA_CUSTOMER_ID=your_customer_id BRIGHT_DATA_ZONE=your_zone_name BRIGHT_DATA_PASSWORD=your_zone_password -
Start Ollama (if not already running):
ollama serve
-
Start Docker for Chroma (if not already running):
docker run -d - name chromadb -v ./chroma-data:/data -p 8000:8000 chromadb/chroma
npm start
# or
node main.jsWhen you first start the chatbot, you'll be in Topic Ingestion Mode. Here you can:
-
Research a new topic by simply typing the topic name:
enter topic to research (or command): ZLUDA -
The system will:
- Generate 20+ targeted search queries
- Fetch comprehensive data using Bright Data SERP API
- Process and chunk the content
- Generate embeddings using local models
- Store everything in ChromaDB
-
Once research is complete, you'll automatically switch to Chat Mode.
In Chat Mode, you can ask any questions about the researched topic:
[ZLUDA] your question: What is ZLUDA and how does it work?
The system will:
- Find the most relevant knowledge chunks
- Generate a contextual answer using the local LLM
- Provide source citations
- Maintain conversation history
list: Show all available topics in the knowledge basestats <topic>: Display statistics for a specific topicswitch <topic>: Switch to Q&A mode for an existing topicclear: Clear conversation historyback: Return to topic selection mode (when in chat)quitorexit: Exit the chatbot
The system is highly configurable through config.js:
- Search Templates: 20+ predefined query patterns for comprehensive research
- Results per Query: Number of search results to fetch
- Request Delays: Rate limiting between API calls
- Chunk Size: Optimal size for text chunks (default: 400 tokens)
- Context Length: Maximum context for LLM generation
- Retrieved Chunks: Number of chunks to use for answering
- Cache Directory: Local cache storage location
- Expiry Time: How long to keep cached search results
rag-chatbot/
├── main.js # Main application and interaction loop
├── config.js # System configuration
├── ollama.js # Ollama client for local LLM
├── search.js # Bright Data SERP API integration
├── vectorstore.js # ChromaDB vector database operations
├── rag.js # RAG system for Q&A
├── cache.js # Caching functionality
├── utils.js # Utility functions
├── package.json # Dependencies and scripts
├── .gitignore # Git ignore patterns
├── env.example # Environment variables template
└── README.md # This file
- Query Generation: Uses predefined templates to create comprehensive search queries
- Data Fetching: Retrieves search results via Bright Data SERP API
- Content Processing: Extracts and chunks relevant information
- Embedding Generation: Creates vector embeddings using local models
- Storage: Stores chunks in topic-specific ChromaDB collections
- Query Embedding: Converts user question to vector representation
- Similarity Search: Finds most relevant chunks from the knowledge base
- Context Building: Assembles retrieved chunks into coherent context
- Answer Generation: Uses local LLM to generate response based on context
- Source Attribution: Provides citations for transparency
- Each topic gets its own ChromaDB collection
- Easy switching between different research areas
- Persistent storage of all researched topics
- Search results are cached to reduce API calls
- Configurable expiry times
- Automatic cleanup of expired cache files
- Maintains recent conversation history
- Uses context for better follow-up responses
- Conversation history per topic
-
Ollama Connection Failed
- Ensure Ollama is running:
ollama serve - Check if models are available:
ollama list
- Ensure Ollama is running:
-
Missing Models
- The system will automatically attempt to pull missing models
- Manually pull:
ollama pull gemma3:4b-it-qatandollama pull nomic-embed-text
-
Search API Errors
- Verify Bright Data credentials in
.env - Check internet connectivity
- Ensure proxy credentials are correct
- Verify Bright Data credentials in
-
ChromaDB Issues
- ChromaDB runs automatically with the Node.js client
- Clear database: delete the
chroma_dbdirectory - Complete ChromaDB Reset: Use the provided PowerShell script for a full reset:
This script will:
.\cleanup_chromadb.ps1
- Stop the running ChromaDB Docker container
- Remove the container and its data volume
- Start a fresh ChromaDB container with clean data
- Useful when you need to completely reset your vector database
- Chunk Size: Adjust based on your use case (smaller = more precise, larger = more context)
- Cache Expiry: Longer expiry reduces API calls but may use stale data
- Search Templates: Customize templates for your specific domain
MIT License - see the original project for license details.

