Skip to content
#

llm-caching

Here are 19 public repositories matching this topic...

Verification layer for semantic caches: checks whether a cached answer is right for the question, not just how similar it is, on top of the cache you already run. Randomized audits give a conservative upper bound on the live error rate; CRC and online ACI thresholds; drift detection with auto-recalibration; entries to refresh; local scoring.

  • Updated Nov 8, 2026
  • Python
CacheVerifier

Research behind CacheVerifier (cacheverifier.com): verifier gating for semantic cache hits vs. static and adaptive thresholds, on ~210k requests across three public benchmarks plus real Twitter support traffic (4 datasets). Fine-tuning vs. off-the-shelf, label noise, drift, conformal risk control under shift, online ACI from randomized audits.

  • Updated Nov 8, 2026
  • Python

Production-ready Retrieval-Augmented Generation (RAG) system using LangChain and Google Gemini. Features hybrid search (Chroma + BM25), reciprocal rank fusion, idempotent document ingestion, persistent SQLite LLM caching, an interactive CLI shell, and a Streamlit UI.

  • Updated Sep 25, 2026
  • Jupyter Notebook

System 0 for your LLM: learns the fixed-choice answers your LLM gives (intents, routing, moderation) and serves the repeats in microseconds from a small CPU table, at most 5% disagreement, certified. Zero-code proxy for the OpenAI Decisions API, System One and chat models.

  • Updated Nov 1, 2026
  • Python

Evaluates prompt similarity via n-gram shingling, token hashing, and min-hash signatures to serve deterministic cached agent responses and eliminate redundant LLM calls | GenPark: https://www.genpark.ai | LinkedIn: https://www.llinkedin.com/company/92955550

  • Updated Oct 30, 2026
  • Python

Add this topic to your repo

To associate your repository with the llm-caching topic, visit your repo's landing page and select "manage topics."

Learn more