feat(aarch64): add GDB debugging for Linux KVM by cshung · Pull Request #2016 · hyperlight-dev/hyperlight

cshung · 2026-11-03T00:34:05Z

ARM64 Linux KVM GDB

KVM guests support register and memory access, two hardware breakpoints,
BRK software breakpoints, and hardware single-step.
KVM reports debug stops through KVM_EXIT_DEBUG.
Debug state survives vCPU reset, and register access respects the active stack pointer.

Commit layers

This PR is C, one KVM-specific commit above shared support A in #2050.
WHP support in #1907 is an independent sibling, not a prerequisite.

KVM-only comparison.

Shared page-table translation, guest-page-sized memory access, snapshot
recovery, cancellation and crash-detach handling belong to #2050.
This layer contains KVM integration, platform enablement and native GDB CI.

The graph and update rules are in
docs/gdb-commit-layers.md.
The full PR diff includes the unmerged shared prerequisite.

Validation

Formatting and ARM64 GDB-enabled cross-Clippy pass with default and KVM-only features.
KVM backend source matches the tested combined tree.
Shared support passes native ARM64 MSHV formatting, Clippy, build, the full
just test recipe, 26 debugger/backend tests and seven real GDB client tests.
Earlier Raspberry Pi 4 validation passed 16 targeted native unit tests and six
real GDB client tests. Native KVM ARM64 runtime tests were not repeated for these
shared fixes.

Pi validation uses a separate, uncommitted ARMv8.0 allocator fix and guests built
with RUSTFLAGS="-C target-feature=-lse". Pi compatibility changes are outside this PR.

Copilot

🟡 Changes recommended

Unresolved critical flaws allow guest modification of executable trampoline code and debugger memory corruption, while breakpoint cleanup and snapshot-ordering issues can leave sandboxes unrecoverable.

5 open issues

What changed in this PR

Adds ARM64 guest debugging on Linux/KVM, alongside the shared debugger infrastructure and Windows/WHP support from #1907.

Changes:

Supports ARM64 register and memory access, software and hardware breakpoints, and hardware single-step.
Adds instruction-cache synchronization, debug-state preservation across reset, and snapshot safeguards.
Adds debugger tests and documents platform capabilities.

File	Description
`src/hyperlight_host/src/sandbox/uninitialized_evolve.rs`	Initializes ARM64 debug vectors during VM setup.
`src/hyperlight_host/src/sandbox/snapshot/memory/backing.rs`	Exposes backing ranges for cache synchronization.
`src/hyperlight_host/src/sandbox/initialized.rs`	Guards snapshots and restores against active software breakpoints.
`src/hyperlight_host/src/mem/mgr.rs`	Exposes checked host memory ranges to the debugger.
`src/hyperlight_host/src/mem/exe.rs`	Carries debug-vector addresses in load metadata.
`src/hyperlight_host/src/mem/elf.rs`	Finds and relocates the debug-vector symbol.
`src/hyperlight_host/src/hypervisor/virtual_machine/whp/x86_64.rs`	Adapts address translation to the shared interface.
`src/hyperlight_host/src/hypervisor/virtual_machine/whp/aarch64.rs`	Implements WHP debugging, cache synchronization, and tests.
`src/hyperlight_host/src/hypervisor/virtual_machine/mshv/x86_64.rs`	Adapts address translation to the shared interface.
`src/hyperlight_host/src/hypervisor/virtual_machine/mod.rs`	Adds ARM64 debug exits and initialization errors.
`src/hyperlight_host/src/hypervisor/virtual_machine/kvm/x86_64.rs`	Adapts address translation to the shared interface.
`src/hyperlight_host/src/hypervisor/virtual_machine/kvm/aarch64.rs`	Implements KVM debugging and active-stack-pointer handling.
`src/hyperlight_host/src/hypervisor/regs/aarch64/whp_reg.rs`	Defines WHP debug register identifiers.
`src/hyperlight_host/src/hypervisor/hyperlight_vm/x86_64.rs`	Uses shared debugger handling and breakpoint storage.
`src/hyperlight_host/src/hypervisor/hyperlight_vm/mod.rs`	Integrates ARM64 stops, stepping, and mapping safeguards.
`src/hyperlight_host/src/hypervisor/hyperlight_vm/debug.rs`	Provides shared requests and software-breakpoint lifecycle handling.
`src/hyperlight_host/src/hypervisor/hyperlight_vm/aarch64.rs`	Connects ARM64 VM setup and execution to GDB.
`src/hyperlight_host/src/hypervisor/gdb/target.rs`	Adds ARM64 register conversion to the shared target.
`src/hyperlight_host/src/hypervisor/gdb/mod.rs`	Adds page-table translation and cache-flush support.
`src/hyperlight_host/src/hypervisor/gdb/event_loop.rs`	Reports ARM64 initial stops to GDB.
`src/hyperlight_host/src/hypervisor/gdb/arch/x86_64.rs`	Houses x86 breakpoint constants and stop classification.
`src/hyperlight_host/src/hypervisor/gdb/arch/aarch64.rs`	Defines ARM64 software breakpoints and alignment checks.
`src/hyperlight_host/src/hypervisor/gdb/arch.rs`	Selects architecture-specific debugger helpers.
`src/hyperlight_host/examples/guest-debugging/main.rs`	Adds ARM64 breakpoint, stepping, and code-write integration tests.
`src/hyperlight_host/build.rs`	Enables GDB for supported ARM64 platforms.
`src/hyperlight_guest_bin/src/arch/aarch64/exception/entry.rs`	Exports debug vectors and forwards synchronous exceptions.
`src/hyperlight_common/src/arch/aarch64/exn.rs`	Defines the shared debug-exception HVC immediate.
`docs/how-to-debug-a-hyperlight-guest.md`	Documents ARM64 capabilities and guest-symbol requirements.
`CHANGELOG.md`	Records Linux/KVM and Windows/WHP ARM64 debugging support.

_{Review effort: Balanced}

hyperlight-gh-bot · 2026-11-05T18:07:34Z

Benchmark Results

Measured commit: 4849110a4736
Baseline commit: a03088470d82

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`773.23 ns` (➖ 1.02x faster)
`vec_bytes`	`579.41 ns` (➖ 1.01x faster)
		`377.47 µs` (➖ 1.01x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`527.99 ns` (➖ 1.01x slower)
`65536`	`143.98 ns` (➖ 1.01x slower)

sandboxes

	`create_initialized_and_drop`
`medium`	`12.92 ms` (➖ 1.03x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`7.72 ns` (➖ 1.00x faster)	`7.69 ns` (➖ 1.00x faster)	`7.73 ns` (➖ 1.00x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`107.03 µs` (➖ 1.08x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`7.50 µs` (➖ 1.08x slower)	`6.98 µs` (➖ 1.05x faster)
`65536`	`1.91 µs` (➖ 1.09x faster)	`2.03 µs` (➖ 1.06x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`26.94 µs` (➖ 1.01x faster)
`65536`	`6.17 µs` (➖ 1.05x faster)	`6.14 µs` (➖ 1.05x faster)
`8192`	`1.02 µs` (➖ 1.03x faster)	`1.04 µs` (➖ 1.02x slower)

kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`690.96 ns` (➖ 1.02x slower)
`vec_bytes`	`514.07 ns` (➖ 1.00x faster)
		`676.67 µs` (➖ 1.02x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`499.74 ns` (➖ 1.00x slower)
`65536`	`156.29 ns` (➖ 1.00x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`14.55 ms` (➖ 1.00x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`6.87 ns` (➖ 1.01x slower)	`6.84 ns` (➖ 1.00x slower)	`6.85 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`48.93 µs` (➖ 1.01x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`7.63 µs` (➖ 1.01x slower)	`7.53 µs` (➖ 1.00x faster)
`65536`	`2.15 µs` (➖ 1.00x slower)	`2.13 µs` (➖ 1.00x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`29.65 µs` (➖ 1.10x faster)
`65536`	`7.20 µs` (➖ 1.01x slower)	`7.12 µs` (➖ 1.01x faster)
`8192`	`725.27 ns` (➖ 1.03x slower)	`731.19 ns` (➖ 1.00x slower)

mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`963.65 ns` (➖ 1.00x faster)
`vec_bytes`	`735.61 ns` (➖ 1.01x slower)
		`301.71 µs` (➖ 1.05x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`759.58 ns` (➖ 1.01x slower)
`65536`	`194.64 ns` (➖ 1.00x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`69.24 ms` (➖ 1.01x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`9.75 ns` (➖ 1.02x faster)	`9.77 ns` (➖ 1.01x faster)	`9.79 ns` (➖ 1.01x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`115.33 µs` (➖ 1.06x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`9.94 µs` (➖ 1.03x slower)	`10.14 µs` (➖ 1.03x slower)
`65536`	`2.25 µs` (➖ 1.01x slower)	`2.42 µs` (➖ 1.03x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`36.74 µs` (➖ 1.06x faster)
`65536`	`8.44 µs` (➖ 1.14x faster)	`9.00 µs` (➖ 1.06x faster)
`8192`	`1.31 µs` (➖ 1.07x slower)	`1.32 µs` (➖ 1.05x slower)

mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`922.56 ns` (➖ 1.02x slower)
`vec_bytes`	`650.62 ns` (➖ 1.00x slower)
		`667.34 µs` (➖ 1.03x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`661.78 ns` (➖ 1.01x faster)
`65536`	`249.45 ns` (➖ 1.08x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`76.16 ms` (➖ 1.04x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`9.66 ns` (➖ 1.10x slower)	`8.70 ns` (➖ 1.00x slower)	`8.90 ns` (➖ 1.02x slower)

snapshot_files

	`load_snapshot_unverified`
`small`	`57.35 µs` (➖ 1.05x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`8.05 µs` (➖ 1.02x slower)	`8.15 µs` (➖ 1.02x slower)
`65536`	`2.33 µs` (➖ 1.00x slower)	`2.30 µs` (➖ 1.01x faster)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`37.55 µs` (➖ 1.04x faster)
`65536`	`7.49 µs` (➖ 1.01x faster)	`7.47 µs` (➖ 1.00x faster)
`8192`	`896.15 ns` (➖ 1.02x faster)	`905.66 ns` (➖ 1.02x faster)

hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.23 µs` (➖ 1.02x slower)
`vec_bytes`	`860.33 ns` (➖ 1.10x slower)
		`1.98 ms` (➖ 1.01x faster)

payload_allocation

	`slot_pool_segmented`
`262144`	`784.26 ns` (➖ 1.01x faster)
`65536`	`230.21 ns` (➖ 1.01x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`14.06 ms` (➖ 1.15x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`9.97 ns` (➖ 1.03x faster)	`10.08 ns` (➖ 1.02x faster)	`9.89 ns` (➖ 1.04x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`828.50 µs` (➖ 1.01x slower)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`9.48 µs` (➖ 1.08x faster)	`8.98 µs` (➖ 1.06x faster)
`65536`	`2.32 µs` (➖ 1.08x faster)	`2.32 µs` (➖ 1.00x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`38.76 µs` (➖ 1.25x faster)
`65536`	`8.86 µs` (➖ 1.02x faster)	`8.76 µs` (➖ 1.01x faster)
`8192`	`1.29 µs` (➖ 1.03x faster)	`1.34 µs` (➖ 1.02x slower)

hyperv-ws2025 / intel (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

	`encode_control`	`decode_vec_bytes_copy`
`byte_chunks`	`1.23 µs` (➖ 1.03x slower)
`vec_bytes`	`802.73 ns` (➖ 1.01x faster)
		`3.14 ms` (➖ 1.03x slower)

payload_allocation

	`slot_pool_segmented`
`262144`	`832.72 ns` (➖ 1.13x slower)
`65536`	`208.02 ns` (➖ 1.01x faster)

sandboxes

	`create_initialized_and_drop`
`medium`	`14.01 ms` (➖ 1.02x faster)

slot_pool

	`alloc_dealloc_128`	`alloc_dealloc_4096`	`alloc_dealloc_1500`
	`9.71 ns` (➖ 1.00x slower)	`10.02 ns` (➖ 1.00x faster)	`9.63 ns` (➖ 1.00x faster)

snapshot_files

	`load_snapshot_unverified`
`small`	`668.43 µs` (➖ 1.01x faster)

virtq_readonly

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`7.60 µs` (➖ 1.02x faster)	`7.64 µs` (➖ 1.01x slower)
`65536`	`2.46 µs` (➖ 1.04x slower)	`2.30 µs` (➖ 1.00x slower)

virtq_readwrite

	`slot_pool_segmented`	`slot_pool_segmented_fragmented`
`262144`	`42.62 µs` (➖ 1.18x faster)
`65536`	`9.04 µs` (➖ 1.07x slower)	`8.30 µs` (➖ 1.01x faster)
`8192`	`1.12 µs` (➖ 1.06x slower)	`1.21 µs` (➖ 1.13x slower)

_{Reported by cargo ci bench-report --candidate run:42074251082 --config-file bench_report.toml.}

Architecture adapters, software breakpoints, page-sized memory access and recovery handling are shared by backend layers. Guest debug vectors and common tests support ARM64. Backend layers control platform enablement. The commit-layer document defines dependencies and shared-update validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>

KVM debug exits provide hardware breakpoints and single-step. Register access respects the active stack pointer. Native ARM64 GDB tests run in debug CI. Shared debugger behavior belongs to the parent commit. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: cshung <3410332+cshung@users.noreply.github.com>

cshung added kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. ready-for-review PR is ready for (re-)review labels Nov 3, 2026

Copilot AI balanced review requested due to automatic review settings November 3, 2026 00:34

cshung requested review from andreiltd, danbugs, dblnz, devigned, jprendes, jsturtevant, ludfjig, simongdavies, squillace and syntactically as code owners November 3, 2026 00:34

cshung marked this pull request as draft November 3, 2026 00:36

cshung removed the ready-for-review PR is ready for (re-)review label Nov 3, 2026

Copilot started reviewing on behalf of cshung November 3, 2026 00:37

View session

Copilot AI reviewed Nov 3, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

cshung force-pushed the cshung/kvm-aarch64-gdb branch from a60be9b to 55da94f Compare November 3, 2026 01:58

This comment has been minimized.

Sign in to view

cshung force-pushed the cshung/kvm-aarch64-gdb branch from 55da94f to 2418379 Compare November 3, 2026 19:28

This comment has been minimized.

Sign in to view

cshung force-pushed the cshung/kvm-aarch64-gdb branch from a26c2d9 to 11361ca Compare November 4, 2026 16:37

This comment has been minimized.

Sign in to view

cshung force-pushed the cshung/kvm-aarch64-gdb branch from 11361ca to 4849110 Compare November 5, 2026 17:24

cshung force-pushed the cshung/kvm-aarch64-gdb branch from 4849110 to 5ddde64 Compare November 11, 2026 20:12

This was referenced Nov 11, 2026

feat(aarch64): add GDB debugging for Windows ARM64 WHP #1907

Open

feat(aarch64): add GDB debugging for Linux MSHV #2052

Open

cshung force-pushed the cshung/kvm-aarch64-gdb branch from 5ddde64 to 3f280ae Compare November 11, 2026 23:24

cshung and others added 2 commits November 11, 2026 16:01

cshung force-pushed the cshung/kvm-aarch64-gdb branch from 3f280ae to 859ef39 Compare November 12, 2026 00:12

Conversation

cshung commented Nov 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

ARM64 Linux KVM GDB

Commit layers

Validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

hyperlight-gh-bot Bot commented Nov 5, 2026

Benchmark Results

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Benchmark Results

function_call_codec

payload_allocation

sandboxes

slot_pool

snapshot_files

virtq_readonly

virtq_readwrite

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

cshung commented Nov 3, 2026 •

edited

Loading