On self-hosted Embed, a snapshot marked success crashes Firecracker on every restore attempt. The previous snapshot of the same sandbox restores normally.
Firecracker panicked at src/vmm/src/devices/mod.rs:34:9:
The number of available virtio descriptors 41919 is greater than queue size: 256!
Verified
- The runtime logged a successful upload and marked the snapshot build
success.
- Restoring the saved snapshot through
Sandbox.create("<env>:default") on a fresh, healthy host with the same binaries produces the same panic 3 out of 3 times. The copied snapshot files and dependencies were hash-verified.
- The previous snapshot, captured 42 minutes earlier, restores and runs guest commands on that host.
The original host logged repeated No space left on device errors before capture, but these logs do not establish the cause. The tests reproduce the restore failure using the preserved snapshot, not the original capture failure.
Expected result
A snapshot marked successful should be restorable. If saving it fails, report the failure and preserve a recovery path for the sandbox.
Environment
- Single-node Embed, local file storage; Ubuntu 26.04 ARM64 VM with nested virtualization
- Orchestrator
v0.16.202609130627-59497eb9134; API v0.14.202609170000-908833e4c12
- Firecracker
v1.14-0.2.0 (reports v1.14.4); hugepage-backed sandbox, 2 vCPU / 2048 MiB
- Compose revision:
a065a4ddb3f2c6a4149634d9acb14b62f65839ac
Snapshot artifacts and logs are preserved. Related: #3658 covers losing a sandbox when pause reports an error.
On self-hosted Embed, a snapshot marked
successcrashes Firecracker on every restore attempt. The previous snapshot of the same sandbox restores normally.Verified
success.Sandbox.create("<env>:default")on a fresh, healthy host with the same binaries produces the same panic 3 out of 3 times. The copied snapshot files and dependencies were hash-verified.The original host logged repeated
No space left on deviceerrors before capture, but these logs do not establish the cause. The tests reproduce the restore failure using the preserved snapshot, not the original capture failure.Expected result
A snapshot marked successful should be restorable. If saving it fails, report the failure and preserve a recovery path for the sandbox.
Environment
v0.16.202609130627-59497eb9134; APIv0.14.202609170000-908833e4c12v1.14-0.2.0(reports v1.14.4); hugepage-backed sandbox, 2 vCPU / 2048 MiBa065a4ddb3f2c6a4149634d9acb14b62f65839acSnapshot artifacts and logs are preserved. Related: #3658 covers losing a sandbox when pause reports an error.