[JonoPrest]

The compiler benchmarks compile runtime and Belt files one at a time and build tests/tests, so costs that grow with the number of modules a build has or loads barely show: the signature copy pull requests (#9192, #9194) moved them by 0.4%, and nothing in them is namespaced. Scheduling changes such as #9191 do the same work sooner, which counting can only show as a cost.

This adds:

• Two generated projects, built with --project like tests/tests (one worker domain, allocated words, clean and after an edit):

  • WideDeps: 240 modules using one or two values of four large modules.
  • Namespace: a "namespace": true package of 1,000 one-line modules.

  Their counts are larger than all the others together, so the report adds them up on a line of their own, Generated projects, and In total covers the other benchmarks as before. Each still has its own row and the 1% threshold.

• time, a local command that builds a generated chain of 150 modules (plus a module beside each) clean with the base's and the head's build systems in turn, 10 rounds by default (--rounds changes it). It prints both medians and the median of the per-round head/base ratios with a bootstrap 95% confidence interval, and writes the times to a JSON file. CI doesn't run it; a pull request that changes the build's scheduling can quote its numbers.

Local results against master:

Benchmark	Change
build/WideDeps/clean, #9194	-34.1%
build/WideDeps/edit, #9194	-37.2%
build/Namespace/clean, namespace map built on demand (#9287)	-66.9%
Chain wall time (local time run), #9191	0.695 (0.646 to 0.825)
Chain wall time (local time run), unrelated change	0.991 (0.948 to 1.055)
Chain wall time (local time run), same build system on both sides	0.965 (0.921 to 1.078)

The other benchmarks' counts are unchanged. The benchmarks job takes about 25 s more for the generated projects, measured locally; a time run takes about 45 s.

Open question: should the generated projects run under --project, as now, or have a flag of their own?

Checklist

• Added a changelog fragment (see changes/README.md), or no release notes are needed and a maintainer should apply changelog:skip.
• Added tests for the fix or feature.
• Gave every commit a description below its subject, saying why it is made (see "Commit messages" in CONTRIBUTING.md).
• Signed off every commit for the DCO (git commit -s).

[JonoPrest]

@codex review

[github-actions]

Changelog

🏠 Internal

• Count the words the build allocates on generated projects in the CI benchmarks: many modules using little of a few large dependencies, and a namespaced package of 1,000 modules.
• Time clean builds of a generated chain of modules with the base's and the head's build systems in turn in the CI benchmarks, and report the median ratio with a 95% confidence interval. The times are only indicative and never fail the job.

Semantic Diff

2 commits · Full page

Count the build's allocations on generated projects in the benchmarks

_{adfb074 · JonoPrest · Nov 09, 2026, 14:22 UTC}

Why

The benchmarks compile runtime and Belt files one at a time, whose dependency graphs are small, and build tests/tests, whose modules use a few small modules. Costs that grow with how many modules a build has or loads barely show: a change that avoids copying the signatures of large dependencies that a module hardly reads moved them by 0.4%, and opening the map of a namespaced package, which is quadratic in its modules per build, doesn't run at all.

With --project, measure now also builds two projects that the script generates, clean and after an edit, on one worker domain and counting allocated words like tests/tests:

- WideDeps: 240 modules that each use one or two values of four modules with 600 types and 1,200 values each. The edit adds a value to one of them, which compiles its users again. - Namespace: a package with "namespace": true and 1,000 one-line modules. The edit changes a module that nothing uses.

Their counts are larger than all the others together, so the report adds them up on a line of their own, Generated projects, and In total covers the other benchmarks as before. Each has its row and threshold like any other benchmark. Every module of a project build loads the runtime's interfaces, so the inputs of the generated projects and of tests/tests now include a hash of the runtime sources: a pull request that changes the runtime marks them as inputs changed rather than as regressions.

Locally, the signature copy pull requests take WideDeps clean from 505M to 333M words and its edit from 232M to 146M, and building the namespace map on demand takes Namespace clean from 611M to 203M; the other benchmarks' counts are unchanged. The builds of tests/tests now share countBuild and editModule with the generated projects.

Time a chain of modules with the base's and the head's build systems

_{511cf97 · JonoPrest · Nov 09, 2026, 14:27 UTC}

Why

A change to the build's scheduling, such as starting a module once the interfaces it reads are final, does the same work sooner. Counting allocated words and instructions can only show what such a change costs, never what it saves.

time <base checkout> <head checkout> <out.json> builds a generated project of 150 modules in a chain, each using the one before it, and a module beside each, clean with the two build systems in turn on the default number of worker domains: one build each that isn't counted, then 10 rounds, the order alternating each round. compare --timings adds a section to the report with the base's and the head's median times and the median of the head's time over the base's in the same round, with a 95% confidence interval from a percentile bootstrap of the rounds with a fixed seed. The section never counts as a regression or an improvement; when the times' file is missing or can't be read, it says they weren't measured. In CI, the step runs after both measurements, each build may take 2 minutes and the whole command 10, and a failure only leaves the times out.

Locally, with load around 5 to 9 on 8 cores, the ratio was 0.695 (0.646 to 0.825) for the pull request that starts dependents once a module's interface is final, 0.991 (0.948 to 1.055) for one that only changes namespaced packages, and 0.965 (0.921 to 1.078) with the same build system on both sides. The command took about 45 seconds.

[chatgpt-codex-connector]

Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-09T14:43:50.029809Z	f5ff1ea	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: f5ff1ea02b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[github-actions]

✅ Compiler benchmarks

Allocated words

In total: 683,548,494 → 683,548,310 (±0.00%).

Generated projects: 1,333,102,129 → 1,333,102,112 (±0.00%).

None of the 265 benchmarks changed by 0.1% or more.

Instructions

In total: 9,664,671,518 → 9,664,671,518 (±0.00%).

None of the 33 benchmarks changed by 0.1% or more.

Output size

In total: 25,569,598 → 25,569,598 (±0.00%).

None of the 13 benchmarks changed by 0.1% or more.

Wall time (indicative only)

Clean builds of a generated chain of modules with the base's and the head's build systems in turn on this runner, on several domains. Wall time on a shared runner is noisy, so this never fails the job; it shows changes such as scheduling ones that the counts above can't. The ratio is the median of each round's head time over its base time, not the ratio of the two medians.

Benchmark	Base, median	Head, median	Median of head / base per round	95% CI
time/Chain/clean (10 rounds)	1225 ms	1226 ms	1.000	0.996 to 1.011

Pull request 511cf97 compared with base b82d944 in this run.

[pkg-pr-new]

Open in StackBlitz

rescript

npm i https://pkg.pr.new/rescript@9282

@rescript/belt

npm i https://pkg.pr.new/@rescript/belt@9282

@rescript/darwin-arm64

npm i https://pkg.pr.new/@rescript/darwin-arm64@9282

@rescript/linux-arm64

npm i https://pkg.pr.new/@rescript/linux-arm64@9282

@rescript/linux-x64

npm i https://pkg.pr.new/@rescript/linux-x64@9282

@rescript/runtime

npm i https://pkg.pr.new/@rescript/runtime@9282

@rescript/win32-x64

npm i https://pkg.pr.new/@rescript/win32-x64@9282

commit: 6cae0e5

[github-actions]

Developer playground preview: https://rescript-lang.github.io/rescript/dev-playground/?version=pr-9282

[github-actions]

Changelog

🏠 Internal

• Count the words the build allocates on generated projects in the CI benchmarks: many modules using little of a few large dependencies, and a namespaced package of 1,000 modules.
• Add a time command to the compiler benchmarks script for local use: it times clean builds of a generated chain of modules with two checkouts' build systems in turn and prints the median ratio with a 95% confidence interval.

Semantic Diff

2 commits · Full page

✏️ Count the build's allocations on generated projects in the benchmarks

_{29b4d05 · JonoPrest · Nov 09, 2026, 14:22 UTC · edited}

Why

The benchmarks compile runtime and Belt files one at a time, whose dependency graphs are small, and build tests/tests, whose modules use a few small modules. Costs that grow with how many modules a build has or loads barely show: a change that avoids copying the signatures of large dependencies that a module hardly reads moved them by 0.4%, and opening the map of a namespaced package, which is quadratic in its modules per build, doesn't run at all.

With --project, measure now also builds two projects that the script generates, clean and after an edit, on one worker domain and counting allocated words like tests/tests:

- WideDeps: 240 modules that each use one or two values of four modules with 600 types and 1,200 values each. The edit adds a value to one of them, which compiles its users again. - Namespace: a package with "namespace": true and 1,000 one-line modules. The edit changes a module that nothing uses.

Their counts are larger than all the others together, so the report adds them up on a line of their own, Generated projects, and In total covers the other benchmarks as before. Each has its row and threshold like any other benchmark, and its inputs are a hash of the project's own files.

Locally, the signature copy pull requests take WideDeps clean from 505M to 333M words and its edit from 232M to 146M, and building the namespace map on demand takes Namespace clean from 611M to 203M; the other benchmarks' counts are unchanged. The builds of tests/tests now share countBuild and editModule with the generated projects.

✏️ Time a chain of modules with the base's and the head's build systems

_{6cae0e5 · JonoPrest · Nov 09, 2026, 14:27 UTC · edited}

Why

A change to the build's scheduling, such as starting a module once the interfaces it reads are final, does the same work sooner. Counting allocated words and instructions can only show what such a change costs, never what it saves.

time <base checkout> <head checkout> <out.json> builds a generated project of 150 modules in a chain, each using the one before it, and a module beside each, clean with the two build systems in turn on the default number of worker domains: one build each that isn't counted, then 10 rounds, the order alternating each round, each build stopped after 2 minutes. It writes the times to the JSON file and prints the base's and the head's median times and the median of the head's time over the base's in the same round, with a 95% confidence interval from a percentile bootstrap of the rounds with a fixed seed. The command is run by hand, so that a pull request that changes the scheduling can quote its numbers; CI doesn't run it.

Locally, with load around 5 to 9 on 8 cores, the ratio was 0.695 (0.646 to 0.825) for the pull request that starts dependents once a module's interface is final, 0.991 (0.948 to 1.055) for one that only changes namespaced packages, and 0.965 (0.921 to 1.078) with the same build system on both sides. The command took about 45 seconds.

[github-actions]

✅ Compiler benchmarks

Allocated words

In total: 682,017,653 → 682,017,015 (±0.00%).

Generated projects: 928,630,889 → 928,630,508 (±0.00%).

None of the 265 benchmarks changed by 0.1% or more.

Instructions

In total: 9,666,376,386 → 9,666,376,386 (±0.00%).

None of the 33 benchmarks changed by 0.1% or more.

Output size

In total: 25,571,598 → 25,571,598 (±0.00%).

None of the 13 benchmarks changed by 0.1% or more.

Pull request 6cae0e5 compared with base 145df8d in this run.

[github-actions]

Developer playground preview: https://rescript-lang.github.io/rescript/dev-playground/?version=pr-9282

[cknitt]

Thanks, the generated projects are a good addition. Costs that grow with the number of modules were the gap in the benchmarks. And in CI, base and head differ by 17 words out of 1.33 G, so they hold up at the 1% threshold.

I'm less sure about the wall-time part. We moved the benchmarks to counts (#9253, #9254) so that nothing in them depends on timings. Could you keep time as a command for local use, but drop the CI step and the --timings section of the report? A scheduling PR can then quote its numbers in its description, like your 0.695 for #9191.

One question on the first commit: with the runtime hash in the inputs, a PR that touches the runtime marks all build/* rows, tests/tests included, as "inputs changed", so a build regression in it no longer fails the check. Did you see a runtime PR move these rows? I'd rather leave the hash out and live with an occasional false alert on an optional job. If they turn out to be common, both sides could build the base's runtime sources, as they already do with tests/tests.

[JonoPrest]

Thanks, the generated projects are a good addition. Costs that grow with the number of modules were the gap in the benchmarks. And in CI, base and head differ by 17 words out of 1.33 G, so they hold up at the 1% threshold.

I'm less sure about the wall-time part. We moved the benchmarks to counts (#9253, #9254) so that nothing in them depends on timings. Could you keep time as a command for local use, but drop the CI step and the --timings section of the report? A scheduling PR can then quote its numbers in its description, like your 0.695 for #9191.

One question on the first commit: with the runtime hash in the inputs, a PR that touches the runtime marks all build/* rows, tests/tests included, as "inputs changed", so a build regression in it no longer fails the check. Did you see a runtime PR move these rows? I'd rather leave the hash out and live with an occasional false alert on an optional job. If they turn out to be common, both sides could build the base's runtime sources, as they already do with tests/tests.

Cool thanks @cknitt

Done on the wall-time part.

Regarding the runtime hash - no I haven't seen a runtime PR move those rows. Came from a codex review concern. Agreed and dropped it.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.93%. Comparing base (145df8d) to head (6cae0e5).

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #9282   +/-   ##
=======================================
  Coverage   81.93%   81.93%           
=======================================
  Files         456      456           
  Lines       65517    65517           
=======================================
+ Hits        53679    53681    +2     
+ Misses      11838    11836    -2     

see 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.