Skip to content

feat: bind calibrations to declared histories - #31

Open
KyleAMathews wants to merge 18 commits into
mainfrom
codex/calibration-history-bindings
Open

KyleAMathews wants to merge 18 commits into
mainfrom
codex/calibration-history-bindings

Conversation

@KyleAMathews

@KyleAMathews KyleAMathews commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Calibrations currently search fixed histories, constructions and generated samples until a mutant applies. Each candidate can open a real database. In the motivating DB release, PowerSync startup calibrations consumed 1,455.762 of the file’s 1,926.534 test seconds.

This change lets a calibration select one declared history. Its clean run must pass and prove its declared reach before the mutant must fail under the named law and optional difference requirement. A failed selected premise fails immediately.

calibrations: [{
  name: 'drops the committed row',
  law: 'durable publication',
  history: { constructed: 'a committed write' },
  corrupt: dropCommittedRow,
}]

Generated sources accept fixed or constructed names. Enumerated sources accept zero-based indices or canonical labels. Invalid selectors fail registration before any driver runs. Fixed and enumerated selections preserve the original identity. Each constructed driver execution builds a fresh history from a stable seed derived from property id and calibration name, then resolves its plan once.

probeHistories binds the automatic controls independently: a clean entered history detects a probe forced not entered; a clean absent history detects a probe forced entered. Entered selection can infer a unique construction; absent selection is explicit. Both paths retain ordinary verdict ranking, so an earlier recorded violation survives a later driver error or controlled-clock spin. Callback failures retain truthful calibration/history context and original cause.

Unbound calibrations retain search during migration. Registration warns once per property/process and names every unbound calibration; collection stays silent. Bound calibrations keep the calibration route. Main’s named campaign/determinism units, UNIT replay and F27 differences are integrated by normal merge. Calibrations still use their stable seed. Fresh calibration seeds, DB migration and eventual search removal remain separate work.

Oracle evidence

The versioned review and repair record preserves every original finding, follow-up, pre-edit gate, mutant result and scope qualification.

  • Law: Authored bindings independently select their original, prove the clean premise and complete declared reach, then reject the designated mutant. Real Vitest receivers preserve registration, warning identity, verdict ranking and truthful callback diagnostics. Successful units retain their own budgets; timed-out work retains correct input/build/resolve relations.
  • Reach: Required constructions cover both forced premises, original identity, metadata, mixed bindings, named seeds, UNIT/replay, per-unit budgets and actual host forwarding. Diagnostic and timeout follow-ups measure ordinary, reordered, seedless and environment receivers separately.
  • RED: Extended oracles reproduced binding/control/metadata defects before source repairs. Later oracle-only repairs reproduced lawful diagnostic false failures and host-dependent timeout observations before changing their judgment/fixture. The record retains historical chronology exceptions explicitly.
  • GREEN: The final combined executable passes 49/49 focused checks; types, lint and formatting pass. Fresh independent diagnostic and timeout reviewers found no new supported defect. Their focused checks pass 18/18 and 24/24; lawful actual source formatting and timing variants pass. Original nine findings and eight latest follow-up gaps are accounted for and repaired within the stated finite scope.
  • Mutants: Actual wrong original/name/cause, environment-only cause omission, extra discarded construction and duplicate/missing input work fail their intended comparisons. Permanent controls preserve late-error, registration, status and process-completeness checks. Setup errors, all-skipped selectors and interrupted runs are never counted as assertion rejections.
  • Limits: Finite synthetic histories and fault/name families under real Node/Vitest. No downstream DB/browser/provider, universal hash-injectivity, entropy-quality or measured release-time improvement is claimed. CHRON-1 remains a historical missed pre-edit gate; later evidence cannot repair its chronology. No budget reduction or dependency/lockfile workaround is included.

Current validation

Final pushed head 81391d3 is green in all CI jobs, including preview publication. The complete framework suite passes 831/831 on Vitest 4.1.12 and 831/831 on Vitest 5.1.0. Each version completes 56 fixed-seed and 56 random-seed units, with nonzero reach for every required diagnostic and timeout receiver. Core/package, plugin, gap oracles, gap end-to-end, type/build/lint/format and packaging checks also pass. Local combined and host-RNG follow-ups pass 49/49 and 18/18. Both final independent reviews and the loss audit found no remaining supported in-scope defect. The PR is ready within the finite contract and historical qualifications in the versioned record. Earlier interrupted local full runs remain interrupted; final complete-suite evidence comes from this exact-head CI run.

Law: An explicit history binding runs only that history. The selected clean
run and actual reach must pass before exactly the designated law rejects the
mutant. Authority: approved maintainer proposal and review, 2026-11-11.

Reach: Constructed framework histories exercise direct selection, invalid
selectors, isolated constructed drivers, and both forced-probe controls.
Fixed seed 1 reached bound-calibration 6 times, registration-error once, and
bound-constructed-driver once. The random campaign reached them 3, 4, and 1
times respectively. Existing run budgets remain unchanged.

RED: On unchanged source at 49f3bd1, the extended framework self-oracle's
'fixed history: bound calibrations use only' test rejected two driver calls
where the binding permits one.

GREEN: 22 focused binding regressions and the selected clean-driver-error
regression pass. Build, lint, types, format, publint, sherif, 56 core tests,
and 4320 ESLint plugin tests pass. Full framework and gap checks are running.

Mutant: The same comparison rejects restoring search (driver count 2 > 1),
skipping reach, accepting an inapplicable corruption, accepting the wrong
law, and cloning a built history that carries identity metadata.

Limits: Unbound calibrations retain search with named registration warnings
until consumers migrate. Fresh seeds await UNIT replay support. DB migration,
a matched DB timing comparison in CI, and the later F27 rebase remain separate.
@changeset-bot

changeset-bot Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 81391d3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@tanstack/oracles Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Pre-merge checks: Passed 1 · Inconclusive 1

Failed Failed checks: 1 inconclusive

Check name Status Explanation Resolution
Title check Pending This assessment has not completed. The check will be retried on the next pre-merge check run.
Passed Passed checks: 1 passed

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@pkg-pr-new

pkg-pr-new Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/TanStack/oracles/@tanstack/eslint-plugin-oracles@31
npm i https://pkg.pr.new/TanStack/oracles/@tanstack/oracles@31

commit: 81391d3

@KyleAMathews
KyleAMathews marked this pull request as ready for review November 11, 2026 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant