Cache LangChain tool results in a Hotdata managed table

[rohan-hotdata]

Summary

LangChain caches LLM calls (set_llm_cache/BaseCache) but has no equivalent for tool
calls -- confirmed by reading the installed langchain_core source directly: neither
BaseTool nor Runnable exposes a cache hook. Every StructuredTool invocation
re-executes, even on an agent retry or a repeated question across sessions.

• HotdataToolCache (hotdata_langchain/cache.py) -- the storage backend. Keys an
  entry by sha256(version + tool_name + args), stores it in a Hotdata managed table
  (cache_key, tool_name, args_json, result_json, created_at) via
  load_managed_table(..., mode="upsert"), reads it back with a plain SQL SELECT.
• cached() -- wraps any plain function or tool callable, not just this package's
  own. Fails open: a broken cache backend degrades to "uncached", never to a broken tool
  call.
• make_hotdata_tools(client, cache=..., cache_ttl=...) wires this into the two read
  tools only (hotdata_execute_sql, hotdata_list_managed_databases) -- the mutating
  tools are never cached, since skipping a mutation on a cache hit would be a correctness
  bug, not caching.

Why now

This was originally blocked: managed tables historically only supported full-table
replace writes (per hotdata-dlt-destination's own docs, which still describe
read-modify-write emulation), which would have made a per-cache-entry write cost
O(cache size). Reading hotdata_framework's actual installed source and runtimedb's git
history directly showed this is now out of date -- native, key-based mode="upsert"
shipped server-side on 2026-07-09. That's what unblocked a simple per-call write path, and
is why this PR also bumps hotdata/hotdata-framework to >=0.8.0 (both released 0.8.0
the same day this was scoped -- newer than Dependabot's open #30/#31, which target the
same version and can likely be closed once this merges) and adds pyarrow as a direct
dependency (already transitive via hotdata-framework; the new module writes parquet
directly).

Verification

• 41/41 tests passing, 15 new in tests/test_cache.py against a fake backend that
  round-trips through real pyarrow parquet writes/reads (not just mocked return values).
  ruff check/format clean; mypy clean on all source files.
• Live-tested against a real workspace this session. Found and confirmed handled: a
  managed table with zero rows ever loaded returns HTTP 400 on query instead of an empty
  result -- cached()'s fail-open logic absorbs this correctly (tool call still
  succeeds), confirmed it doesn't recur once the table holds at least one row.
• Live LangGraph benchmark against real TPCH sf=1 data (6M-row lineitem, a 6-way join
  for revenue-by-nation) showed 3.5-4.8x faster over 12 calls once the cache warms up;
  cold-write cost (~4-6s) breaks even in 2-3 repeats for queries at this scale. Trivial
  queries (SELECT 1) don't clearly benefit -- a cache lookup has its own ~0.6-0.7s
  floor, since it's still a network round trip. This scoping (expensive + repeated, not
  cheap + occasional) is called out in the new README section.

Known limitations (documented in README)

• Managed-database names aren't unique/identifying -- concurrent first-use across
  processes can each create a distinct database with the same name, silently splitting
  the cache. Mitigation: pass database_id= to pin one explicitly.
• All cache writes funnel through one per-table write lock regardless of key.
• HotdataToolCache is memoized per-instance -- construct one per process and reuse it.

Not in this PR

Semantic/fuzzy cache matching (embedding-based similarity for near-duplicate calls) is a
separate follow-up that would reuse Hotdata's vector-index infrastructure; not a
dependency of this change.

Status

Draft -- not ready to merge. Opening for early visibility/feedback.

🤖 Generated with Claude Code