[heft-native] Round 4: SIMD + SHA-NI + syscall squeeze (experiment) - #6099
Draft
Sean Larkin (TheLarkInn) wants to merge 1 commit into
Draft
Sean Larkin (TheLarkInn) wants to merge 1 commit into
Sean Larkin (TheLarkInn) wants to merge 1 commit into
Conversation
Stacked on the Rust rewrite. Pure apps/heft-native changes. - SIMD JSON byte-scans (SSE2/AVX2, src/simd) behind runtime CPU detection with a scalar fallback and a kill switch. Isolated 8.6x at 256 B, up to 26x on 4 KB runs, 10.3x on CJK text. - SHA-NI file hashing and forked workers for large trees: 64 MiB asset up-to-date 6.1x, 10k-file clean 3.9x vs the Rust base. - Syscall cuts on the Tier 0 path: heft --help 211 -> 158 syscalls, kernel time -26%. Static glibc link. - Tier 0 wall 1.40-1.55x, peak RSS -48..-50% vs the Rust base; heft --help 2.0 ms (203x vs heft 1.3.1). Node-plugin tiers unchanged (Node-bound). Output byte-identical, exit codes too. std only, no crates, no Node bindings. unsafe only in src/sys and src/simd. ~285M SIMD-vs-scalar fuzz cases, 0 mismatches. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Squeeze the Rust heft to the metal. Assembly-class optimizations, same output.
Stacked on #6098, so this diff is the Round-4 work only (apps/heft-native). Full diff vs main: main...thelarkinn/heft-native-rust-r4
The honest ceiling
A no-libc
write()+exit()binary takes ~0.4-0.58 ms here: the kernel's floor for any process.heft --helpis ~2 ms, so 10x on startup wall time (0.2 ms) is below the floor and impossible. This PR chases the floor and puts the real 10x-class wins where the work actually is.Highest wins
--clean--help/--version/native build)heft --helpvs heft 1.3.1What changed (apps/heft-native only)
src/simd): SSE2/AVX2 JSON byte-scans behind runtimeis_x86_feature_detected!, with a scalar fallback and aHEFT_NATIVE_SIMD=0kill switch. Portable intrinsics, no raw asm.src/builtin).--help211 -> 158 syscalls, stat-like 83 -> 54.main.Safety and compatibility
unsafeonly insrc/sysandsrc/simd; std only, no crates, no Node bindings.Limits
RUSTFLAGS="-C target-feature=-crt-static".