Repository navigation
test(oracle-gaps): bound the e2e tail; one edit sequence over the witness module - #39
Merged
Merged
Conversation
…ness module A generated history with the witness module takes no steps: each step adds a cold run of the witness module, and about one campaign in six drew a history past the test's limit. The constructed history "an edit sequence over the witness module" (test, helper, revert) keeps law 9 judged over the witness module's outcome classes in every campaign, and enters the new required state "a timed-out run reused after an edit". The grammar narrows through a map with an unmapper, so fast-check still takes every history back. The limit is stated in the header and in law.limits (maintainer decision, 2026-11-11, option M). Held until D: the construction costs about 160 s locally, and its first fixed-seed run found an intermittent law 9 violation (a stale daemon kill after the helper edit), which is fixed separately. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
The constructed history "an edit sequence over the witness module" is a fixed-seed, a random, and a determinism unit. They go to e2e-4, e2e-5, and e2e-1, measured at 97 s, about 97 s, and 189 s locally under load. That puts those shards near 620 to 710 s in the measured run's terms, past the packing's 522 s, so the shards need a new longest-first packing from a full run before CI runs them. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
With the witness module's edit sequence (85 s locally) and the generated witness histories now taking no steps, the fixed-seed units cost 245 s; the two edit sequences are 130 s of it, the other 30 units 115 s. Co-authored-by: Isaac <no-reply@databricks.com>
|
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
 |
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The end-to-end gap oracle no longer generates a history that has both the witness module and law 9's edit steps. One constructed history keeps that combination in every campaign. This bounds the slow tail of the random units. It also makes every campaign judge law 9 over the witness module's outcome classes. The fixed-seed units that a pull request runs get cheaper too: 245 s locally, against 325 s on
main.Why the tail was long
The witness module holds the functions whose outcome classes generated histories rarely reach: hangs, slow kills, crashes, and framework errors. A cold run of a package with it takes about 20 s locally. Each edit step adds a cold run and a daemon check. A sample of 50,000 histories from the old grammar showed this:
On CI that is more than 10 minutes for one history.
The same mechanism made some fixed-seed units slow. A constructed history is built from a generated history, and after #37 two of them drew a base with the witness module and steps. "A declaration fault: missingSource" took 89 s, and "a declaration fault: brokenVitest" took 47 s.
What changes
law.limitssay that law 9 judges the witness module's outcome classes along this one sequence only. They name this oracle as the owner of the gap and the maintainer decision of 2026-11-11.Cost and shards
The fixed-seed units, measured locally under the e2e lock:
main(908b1f9)The two CI shards of the fixed seed (
scripts/pr-shards.mjs) are repacked. The first takes the two edit sequences, and the second takes the other 30 units. Run as CI runs them, they took 140 s and 119 s locally. The shards' check passes, and the shards together select exactly what-t "fixed seed"selects.The future full matrix (
scripts/e2e-shards.mjs) puts the new history's three units ine2e-1(determinism, 189 s),e2e-4(fixed seed), ande2e-5(random seed). Its check passes. Those shards are now above the 522 s of its packing, so they need a new packing from a full run before CI runs them.What the history found
The history's first fixed-seed run found an intermittent law 9 violation. After the helper edit, the daemon reported a stale kill. #37 fixed that defect. With #37, this history's fixed-seed unit and determinism unit pass.
Verification
This pull request and its description were written by Isaac.