Repository navigation
feat(llma): support scorer and experiment tags - #1085
Conversation
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
There was a problem hiding this comment.
Not approved — this change needs a human reviewer.
Re-adding the stamphog label gives the same result unless the changed files change.
stamphog won't auto-review this pull request because two gates refused it. The deny-list gate flagged it for touching the public API: it changes Scorer, adds tag parameters and get_tags()/set_tags() methods, and updates references/public_api_snapshot.txt. The tier gate classified it as T2-never, meaning a cross-cutting feature (395 lines across 9 files) that is not eligible for automated review. The size gate passed, so splitting the pull request would not clear the public API or tier refusals.
The author should ask a human reviewer to look at it. The PR description already notes that human review is required, so this is the expected path.
- greptile-apps[bot] reviewed the current head.
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✗ | matches: public_api |
| size | ✓ | 147L, 4F substantive, 395L/9F incl. docs/generated/snapshots — within ceiling |
| tier | ✗ | classified as T2-never: T2-never (395L, 9F, cross-cutting, feat) |
| stamphog 2.6.0 | .stamphog/policy.yml @ unknown · reviewed head 484c415 |
posthog-python Compliance ReportDate: 2026-10-19T18:13:50.791749+00:00 ✅ All Tests Passed!121/121 tests passed Capture_V1 Tests✅ 95/95 tests passed View Details
Capture_Ai Tests✅ 5/5 tests passed View Details
Feature_Flags Tests✅ 17/17 tests passed View Details
Feature_Flags_Local_Evaluation Tests✅ 4/4 tests passed View Details
|
💡 Motivation and Context
The app supports tags for scorers and offline experiments, but Python SDK callers cannot manage them. This adds matching synchronous and asynchronous public APIs:
Scorer.tags, andany/alltag filters on scorer lists.get_tags()andset_tags()methods for reading, replacing, and clearing tags, including after completion.Tags remain separate from immutable scorer versions and experiment upload declarations. Usage examples document replacement behavior and personal-key permissions.
💚 How did you test it?
📝 Checklist
🤖 Agent context
Autonomy: Human-driven (agent-assisted). DRI: @Radu-Raicea.
Implemented with Codex using Git, GitHub CLI, pytest, Ruff, and mypy. Local Codex session. The implementation follows the existing app API and preserves compatibility with older scorer responses that omit tags. Requires human review.