Repository navigation
feat(llma): support composite offline scorers - #1235
Open
Radu-Raicea wants to merge 1 commit into
Open
Radu-Raicea wants to merge 1 commit into
Radu-Raicea wants to merge 1 commit into
Conversation
PR overviewNo security issues were found in this pull request. Reviews
|
7 tasks done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
💡 Motivation and Context
Offline evaluation callers can create and version composite scorers, validate formulas, and request per-item calculations through the existing sync and async clients. Public types expose pinned inputs and dependency metadata; upload receipts expose stored outcomes, including calculation errors.
Bulk uploads keep composite items and their supplied inputs together while preserving order and exact retry recovery. Ordinary uploads retain their existing wire format. Includes documentation, a replayable example, and a minor release changeset.
Requires the backend rollout from PostHog/posthog#132949. The accompanying SDK contract update defines the new behavior; no intentional contract divergence.
💚 How did you test it?
📝 Checklist
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Codex assisted implementation and validation using Git, GitHub, uv, pytest, Ruff, mypy, and Pyright. Composite grouping is conservative because ingestion-only credentials cannot discover dependencies; oversized groups fail before any upload. Validation used mocked API contracts and inspected backend source, without a live-backend calculation test.