Skip to content

[SC-18115] Add lightweight metrics SDK - #556

Open
cachafla wants to merge 4 commits into
mainfrom
cachafla/sc-18115/lightweight-metrics-sdk
Open

[SC-18115] Add lightweight metrics SDK#556
cachafla wants to merge 4 commits into
mainfrom
cachafla/sc-18115/lightweight-metrics-sdk

Conversation

@cachafla

Copy link
Copy Markdown
Contributor

What and why?

Adds a dependency-light validmind-metrics SDK and shared validmind-tracking-core package for applications that only need to send metrics. The existing library now uses the shared synchronous metric transport, while its async metric API offloads the blocking request instead of creating an aiohttp session. This removes nested event-loop and closed-session failures when metrics are sent from HTTP handlers, while preserving the full library's other async operations.

How to test

  • make lint
  • uv lock --check
  • uv build --all-packages --out-dir /tmp/validmind-library-build-3
  • uv run --package validmind-tracking-core python -m unittest discover packages/validmind-tracking-core/tests
  • uv run --package validmind-metrics python -m unittest discover packages/validmind-metrics/tests
  • uv pip check in a clean virtual environment after installing the built validmind-metrics wheel
  • Root unittest discovery reported 339 tests passing; the local process lingered during existing headless-browser/PostHog shutdown after the suite completed.

What needs special review?

  • OIDC device-flow, credential-cache, refresh-token, and bearer-token behavior in validmind-tracking-core.
  • The async boundary: metric requests use requests in a worker thread and do not share the full library's aiohttp session.
  • Independent package release ordering and the runtime dependency from validmind to validmind-tracking-core.

Dependencies, breaking changes, and deployment notes

  • Adds workspace distributions validmind-tracking-core and validmind-metrics.
  • The existing validmind wheel now depends on validmind-tracking-core; publish the core package before releasing a root-library version containing this dependency.
  • New release workflows use tracking-core-v*.*.* and metrics-v*.*.* tags.
  • No database migrations or new environment-variable names are introduced. Existing VM_API_* and VM_OIDC_* configuration is supported by the lightweight client.
  • Existing validmind.log_metric() and validmind.alog_metric() APIs remain available.

Release notes

Added a lightweight validmind-metrics SDK for sending ValidMind metrics from services without installing the full ML/testing stack. It supports API keys and OIDC authentication and is safe to use from async HTTP handlers.

Checklist

  • What and why
  • Screenshots or videos (Frontend) (N/A — no frontend changes)
  • How to test
  • What needs special review
  • Dependencies, breaking changes, and deployment notes
  • Labels applied
  • PR linked to Shortcut: sc-18115
  • Unit tests added (Backend)
  • Tested locally
  • Documentation updated (if required)
  • Environment variable additions/changes documented (if required)

@cachafla cachafla added enhancement New feature or request dependencies Pull requests that update a dependency file python Pull requests that update Python code security labels Aug 27, 2026
@cachafla
cachafla requested a review from nibalizer August 27, 2026 08:06
@cachafla
cachafla marked this pull request as ready for review August 27, 2026 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file enhancement New feature or request python Pull requests that update Python code security

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants