Skip to content

display: skip idle frames on the O2 and VPro with a shared screen-changed flag - #698

Draft
jamesbraid wants to merge 4 commits into
mainfrom
wgpu/headless-ci
Draft

display: skip idle frames on the O2 and VPro with a shared screen-changed flag#698
jamesbraid wants to merge 4 commits into
mainfrom
wgpu/headless-ci

Conversation

@jamesbraid

@jamesbraid jamesbraid commented Nov 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

The display threads wake sixty times a second whether or not the guest drew anything, and on two boards that costs real CPU: an idle O2 desktop burns about 860 G user instructions a minute in O2-Display, and a V12 Tezro with the GPU raster unit about 21 G in odyssey-display, for a picture nobody is looking at in ci mode. This takes the maintainer's rule from the review (compose on processed commands or display register writes, not on framebuffer writes) and applies it where it was costing, using one shared flag, and leaves the other boards' loops alone.

This replaces the earlier version of this PR. Its offscreen wgpu presenter, the lavapipe/Windows/Metal CI jobs and the VPro compose rule that rode on them are not in here; they stay as a branch and come back as separate changes if you want them.

Four commits:

  • gfx_display: add a screen-changed flag and a frame wait for display loops. ScreenChanged (release mark, acquire-release take, own cache line, one store per processed command or register write), FrameWait (the fixed-start 60 Hz wait a loop already has, plus a park until a mark, a wake or the board's own timer, coalescing marks inside one vsync), FrameRequests (KONA's counted frame request, lifted so a second board can use it; KONA switches to it with no change), and screen frames [reset] on the monitor: frames, composed, skipped. No loop is ported.
  • ip32: compose the display only when the screen changed. The O2's frame buffer is RAM, so the marks are the rendering engine finishing a primitive that wrote memory (once per primitive, not per pixel), a GBE register write that changes the register, a shadow control going live at blanking, and restored state. GBE decode now writes the final 0xAABBGGRR word, which removes the per-frame pack pass (24.9 M instructions a frame in a standalone copy of the loop, against 0.5 M for the row copy that replaces it).
  • odyssey: compose no frame while nothing shows or reads one. --ci with no window and no screenshot, frame request or VNC client: the retrace still runs and the size is published, nothing is composed or read back.
  • ng1: mark the frame buffer after the entry's effect. The REX3 processor marked before applying a GFIFO entry, so a frame that took the mark during a GO composed without that draw and nothing marked again: the last draw of a burst could stay off screen while the guest idled. One store per entry, moved after the effect.

What to look at

  • The park/wake handshake in gfx_display.rs: mark() is one release store plus a relaxed load of parked. A mark that races the thread into its park can miss the unpark, so the thread rechecks every 100 ms; that costs up to 100 ms of latency on that update, not a frame. Say if you would rather the writer pay a fence. Newport does not use it: its flag stays a plain AtomicBool with the store moved after the effect.
  • The O2 rule. A guest that stores into the frame buffer with the CPU and issues no command would show late, because the hardware gives no signal for it. I checked rather than assumed: an audit build scanned every unmarked frame and compared it with the last composed one through boot, the greeter, typed characters and Enter on IRIX 6.5.30. 12,600 idle frames, one differed, for one frame, in the login fields, which is the engine's pixels being visible a few microseconds before its mark (a CPU store would have stayed different). That covers Xsgi at the greeter, not every program, and not Linux, NetBSD, OpenBSD or the PROM console. So that a guest that stores with the CPU and issues no command (a Linux frame buffer console) is not stale for long, a watched O2 screen (a renderer, or a VNC client) is scanned at least once a second whatever marked it; with nothing watching the thread parks. A capturing VNC client still makes the O2 scan every frame, as before; say if you would rather it ride on marks plus the one-second rescan.
  • Gbe::write_reg does not mark when a register is written with the value it holds. IRIX's retrace handler rewrites the same overlay and frame-control shadows and the same interrupt line every frame; marking those had the display compose 1195 of 2457 idle frames in a first audit run.
  • The VPro skip: frames are skipped while nothing consumes them, and a request composes the next frame whole from board memory and registers as they are then. The readers of rgba, the size and frame_seq (ci socket, screen commands, VNC, pointer steering) are listed in the commit message.

Evidence

x86-64, thin-LTO release build, IRIX 6.5.30 at the greeter in ci mode with no window. Display-thread instructions:u by perf stat -t on the named thread, 60 s idle samples, boots ordered main, branch, branch, main, two samples per boot. Main is upstream 4921038 and the branch is these four commits on it.

main branch
O2, idle minute 861.3, 861.9, 856.6, 863.2 G 1.363 G in all four
O2, 30 s typing 12 characters 427.5, 433.9 G 4.43, 4.77 G
Tezro V12, GPU raster, idle minute 21.42, 21.42, 21.41, 21.41 G 3.255 to 3.257 M
Tezro V12, CPU raster, idle minute 3.771 to 3.772 M 3.255 to 3.256 M
Indy (Newport), idle minute 506.7 to 507.6 M 506.7 to 507.6 M

A frame on the O2 costs 340.7 M instructions, so the branch's idle figure is four composes a minute (screen frames); the main figure varies with host speed because a frame takes longer than its period. Screenshots at the greeter, after typing and after Enter hash the same on main and the branch on every boot, for O2, Indy, and Tezro with both raster units. odyssey::raster_bench instruction counts, four alternating runs on the previous base: FlatFill -0.10%, SmoothDepth -0.21%, Lines -0.05%, Clear +0.37%, BigQuads -0.15%.

Checks: cargo ci-test (jitv2, rex-jit, gputrace) 5683 passed, 74 skipped; cargo check --all-targets with default features, with --no-default-features and with --features wgpu; each commit builds. Plain cargo test on this machine fails tests that write fixed paths in a shared /tmp (29 on main, more on the branch in the same run); the ones that failed on the branch and not on main all pass when rerun with --test-threads=1.

Not verified: arm64 (the marks are release stores, stlrb there); a real window on any board; the REX3 corpus replay, which is built only with rex-gpu and does not use the flag that moved; guests that draw into the O2 frame buffer with CPU stores. screen frames [reset] is in debug.md and the CHANGELOG entries are in the commits. The Newport fix is unrelated to the skipping and can go as its own PR if you prefer.

@jamesbraid
jamesbraid force-pushed the wgpu/headless-ci branch 2 times, most recently from fce18a5 to e193f5f Compare November 13, 2026 08:55
@techomancer

techomancer commented Nov 13, 2026

Copy link
Copy Markdown
Owner

counting every framebuffer write is very costly. and possibly incorrect. we dont really know if the pixels written during rendering are actually used by next frame or not, because the hardware could very well render to pbuffer
we should trigger compose if any command has been processed for raster. or if any display register has been updated. this is much simpler logic. runs at much lower frequency and is independent on whether gpu or cpu rendering or composition is used.

@jamesbraid
jamesbraid marked this pull request as draft November 14, 2026 02:07
@jamesbraid
gfx_display: add a screen-changed flag and a frame wait for display l…
a2d028e 
…oops

Each board's display thread wakes sixty times a second whether or not the
guest drew anything, and the boards that skip unchanged frames each keep
their own flag with their own memory ordering. This adds the two parts a
loop needs to do better, without replacing any loop.

ScreenChanged is the flag: a release store to mark, an acquire-release swap
to take, always stored (a load-and-skip would lose a draw to a frame that
takes the flag in between), on a cache line of its own. A writer marks once
per processed command, job or display-register write, never per pixel.

FrameWait is the wait at the end of a loop's frame. wait() sleeps out the
period from a fixed start and resyncs after a stall; park_until_marked()
parks until a mark, a wake or the board's own timer, and then starts the
frame at the next vsync, so marks between two vsyncs make one frame. A mark
costs one relaxed load more than a plain flag store, of whether the thread
is parked. That load cannot be ordered against the store without a barrier
on the writer's path, so a mark racing the thread into its park can miss the
unpark; the thread looks again every 100 ms, so that costs latency, not a
frame.

FrameRequests is KONA's counted frame request lifted out of its display, for
any board to use: a request is pending until a frame composed after it is in
the screen, so one made while a frame is being composed is not lost. A
screenshot, `request_frame` and a capturing client are each one. KONA now uses
the shared type, with no change in what it does.

FrameCounts and `screen frames [reset]` let the monitor say how many frames
a board's display thread ran, composed and skipped, which until now could
only be seen as the thread's share of a host core. A board that counts them
returns the counts from GfxDisplay::frame_counts; the rest say so.

The wait, the wake and the counts have their first users in the commits after
this one; only KONA uses the request type here.

Tests: the mark and take, the counts, a request made mid-frame staying
pending, nested captures, pacing without drift, resync after a
stall, a wake ending a wait and a park, a mark waking a parked thread and not
before, marks inside one vsync making one frame, a mark after a long park
starting the frame at once, and the board's timer.
This was referenced Nov 14, 2026
Merged
Merged
@jamesbraid
ip32: compose the display only when the screen changed
0306213 
The O2's display thread scanned GBE out of RAM and presented the result
sixty times a second whether or not anything had changed. One scan is
340.7 M user instructions (the idle minutes below differ by exactly that
per extra frame), so an idle IRIX greeter cost the thread 857 to 863 G
instructions a minute, some 2500 frames rather than 3600 because a frame
takes longer than its period. After the scan a second pass packed each
pixel from RGBA bytes into the 0xAABBGGRR words the renderers take and
stored them twice, 24.9 M instructions a frame in a standalone copy of
the loop, against 0.5 M for a row copy.

The frame buffer is RAM, so there is no write to hang a mark on, and a
bare store by the CPU into it carries no command. What changes the
picture, and so marks it now:

- the rendering engine finishing a primitive that wrote memory, marked
  once per primitive after its pages are written (RamWindow::end), not
  per pixel;
- a GBE register write that changes the register: the colour maps, the
  cursor, the tile lists, the timing. IRIX's retrace handler rewrites the
  same overlay and frame control shadows and the same interrupt line
  every frame; counting those as changes had the display compose 1195 of
  2457 idle frames in a first audit run;
- a shadow control going live at vertical blanking, which the display
  asks about (latch_pending) because a parked thread never looks;
- restored GBE state or memory.

Whether a guest also draws with CPU stores and no command was checked
for IRIX, not assumed: an audit build scanned every unmarked frame and
compared it with the last composed one, through boot to the greeter, 12
typed characters, three more and Enter on IRIX 6.5.30. In 12,600 idle
frames the screen differed from the last composed one in one, for one
frame, 8550 pixels in the login fields. A CPU store would have stayed
different until the next compose, so that frame is the engine's pixels
being visible a few microseconds before its mark. Xsgi on this machine
draws through the rendering engine, as a count in an audit run shows:
15,574 engine marks against 292 cursor-register writes in a minute. Not
checked: Linux, NetBSD, OpenBSD or the PROM console. A guest that stores
straight into the frame buffer (a Linux frame buffer console) has no
signal the hardware gives, so while a renderer or a capturing client
watches, the screen is scanned at least once a second whatever the marks
say, which bounds how stale such a picture gets; with nothing watching the
thread parks, and only a request or a mark brings a frame.

GBE decode now writes the final word, alpha set, so the display copies
rows into the screen's two buffers and the pack loop is gone. The
control port's shot reads the same words and writes them through
save_png.

A window still has its unchanged frame presented every sixth frame, so
the status bar moves, and a renderer that arrives gets a frame at once.
With nothing watching, the thread parks until a mark, a request or a stop,
waking every 250 ms to look, and every period while a shadow control is
still to go live. Screenshots and frame requests get a fresh frame, using
the counted request type, and a request wakes the thread; a capturing
client gets one every period. `screen shot` waits up to 250 ms for its
frame, and a scan costs 340.7 M instructions, so on a loaded host it can
report an old frame; the O2 did not honour requests before this.

Measured on IRIX 6.5.30 at the greeter, display-thread instructions:u,
release build with thin LTO, ci mode with no window, boots alternating
main, branch, branch, main, two 60 s samples each:
- idle minute: main 861.3, 861.9, 856.6, 863.2 G; branch 1.363 G in all
  four (four composes of 340.7 M)
- 30 s typing twelve characters: main 427.5 and 433.9 G; branch 4.43 and
  4.77 G
- screenshots at the greeter, after typing three characters and after
  Enter hash the same on all four boots of both builds.
Only this x86-64 host ran. arm64 is unmeasured: the marks are release
stores, once per primitive or register change.

Tests: a register write marks and a read does not; a shadow going live at
blanking marks, and one rewritten with the live value does not; restored
state and memory mark; a primitive marks once, after its pixels, and its
register writes mark nothing; the display composes for a register write, a
memory mark, a request and a capture and for nothing else, with the counts
saying so; and a fixed scene (32-bit pixels through the gamma map, a
random colour map, an overlay and the cursor) hashes to the value
recorded on main before the format changed.
@jamesbraid
odyssey: compose no frame while nothing shows or reads one
37c6361 
In ci mode with no window the VPro display composed and read back a frame
sixty times a second for nobody. With the GPU raster unit that is a
scan-out, a readback and a pass over every pixel: 21.4 G
display-thread instructions in an idle minute at the IRIX greeter of a V12
Tezro. With the CPU raster unit the compose already skipped unchanged
frames, and the minute cost 3.77 M.

A frame is read by a renderer (the window, or any renderer installed), by
a screenshot or frame request, or by a client that captures every frame
(VNC). With none of those, the display now runs the retrace, which the
guest waits on, publishes the screen's size so `screen info` and the
pointer steering have it, and composes nothing. Before this change each of
those consumers' readers was checked: the control socket's screenshot and
the monitor's `screen shot` and `screen pix` ask for a frame
(wait_fresh_frame) and wait for it; `screen wait` and the VNC grab loop
do the same or turn capture on; `screen info` reads the size and the
frame count, which a skipped frame leaves where they were. A request made
while frames are skipped composes the next frame whole, from board memory
and the registers as they are then, since nothing consumed the marks and
generations meanwhile, and `rgba` holds it before the request clears.

The frame counts are kept for the monitor: frames skipped and composed.

Measured on a V12 Tezro (2 CPUs, IRIX 6.5.30) at the greeter,
display-thread instructions:u, ci mode with no window, thin-LTO release
build, boots alternating main, branch, branch, main, two 60 s samples:
- GPU raster, idle minute: main 21.42, 21.42, 21.41, 21.41 G; branch 3.255
  to 3.257 M
- CPU raster, idle minute: main 3.771 to 3.772 M; branch 3.255 to 3.256 M
- 30 s typing: main 9.85 and 10.71 G (GPU raster), 0.31 and 0.37 G (CPU);
  branch 1.63 M in both
- greeter, typed and Enter screenshots hash the same on all four boots of
  both builds, for both raster units.
odyssey::raster_bench instruction counts are unchanged (FlatFill -0.10%,
SmoothDepth -0.21%, Lines -0.05%, Clear +0.37%, BigQuads -0.15%, four
alternating runs; the change is outside the draw path). The windowed path is
untouched, and a window with the GPU renderer still takes the unchanged
frame skip.

Tests: nothing composed, size known and no frame presented; the retrace
still runs; a request composes the next frame from what was drawn while
skipping, and rgba holds it when the request clears; capture and a
renderer bring frames back, and leaving takes them away. Tests that run
the display thread's frame to compare pictures turn capture on first, as
a client reading every frame would.
@jamesbraid
ng1: mark the frame buffer after the entry's effect
1741770 
The REX3 processor marked the frame buffer changed when it peeked a GFIFO
entry, before applying the entry's register write or running its GO. A
display frame that took the mark while a GO was still drawing composed
without that draw, and nothing marked again until the next GFIFO entry.
An idle frame does not compose, so the last draw of a burst could stay
off the screen until the guest next touched the board.

The store now follows the entry's effect, and is a release store, so a
frame that takes the flag reads the pixels it follows. It is still one
plain store per entry, as before: the flag stays an AtomicBool, with no
park or wake state on the processor's path. The REX3 corpus replay
instruction count was not rerun: that test is compiled only with rex-gpu,
which marks through the damage tracker, not through this flag.

Test: a hook parks the processor with a GO peeked and not applied, the
test plays the display's frame (take the flag, read the pixels), releases
the GO and checks the flag is up afterwards. It passes on every run. With
the store moved back before the hold point, the old order, it fails 3 of
3 runs: the flag was taken during the hold, the draw landed after the
frame and nothing marked it again.

Newport's idle display-thread cost on an Indy at the greeter is unchanged:
506.7 to 507.6 M instructions a minute on main and on the branch (four
boots each, alternating, two samples a boot), and the greeter, typed and
Enter screenshots hash the same on all four.
@jamesbraid jamesbraid changed the title odyssey, wgpu: compose the GPU display only on change, and run that path headless in CI display: skip idle frames on the O2 and VPro with a shared screen-changed flag Nov 14, 2026
@techomancer

techomancer commented Nov 14, 2026

Copy link
Copy Markdown
Owner

aww the O2 direct memory writes seem like pain in the butt for correctness. but as long as we keep this optional for now it is OK, laptop users can suffer some delayed screen. also we can consider forced refresh once a second or something like that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@jamesbraid @techomancer