Skip to content

An array is a reference to a header that never moves (#453, #477) - #480

Merged
ASDAlexander77 merged 18 commits into
mainfrom
array-reference-switch
Oct 4, 2026
Merged

ASDAlexander77 merged 18 commits into
mainfrom
array-reference-switch

Conversation

@ASDAlexander77

Copy link
Copy Markdown
Owner

PR 2 of 3 for arrays as references (spec: tslang/docs/superpowers/specs/2026-10-03-array-reference-design.md, plan: tslang/docs/superpowers/plans/2026-10-03-array-reference.md). PR 1 (#478, the ArrayLayout helper) is merged; PR 3 (amortised growth) follows.

The representation

An array value (T[], the typed-array aliases, string[]) is now one pointer to a heap header { data, length, capacity } that never moves; only data is reallocated. Assigning, passing, returning, storing, capturing and boxing copy the pointer, and === compares headers, so push, pop, splice and length = through any name are seen through every name (#453). Capacity equals length in this PR.

  • gc: header and data from GC_malloc. none: nothing freed, one more allocation. rc: the header is the counted block; it holds one counted reference to its data block (see the amendment below). own: the header is a single-owned block; a borrowed array parameter is the owner's header, so a push through it is the owner's. Own's rules are unchanged (aliases stay compile errors).
  • The ABI of every function taking or returning an array changes; objects built before this PR do not link with objects built after it.
  • Debug info: an array is a pointer to a { data, length, capacity } struct.

Rulings

  • R1 - null header = empty array. A null header (zeroed element of new Array<T[]>(n), a field with no initializer, undefined cast to an array) reads as an empty array; an op that changes an array through its slot stores a fresh empty header into a null slot first. Documented edge (spec section 3): a null slot passed by value to a parameter (an unset field, an element made by length =) - a push through the parameter materialises a header in the parameter's copy, so the slot's owner does not see it and, under rc and own, that header leaks. Accepted: such a slot holds undefined in TypeScript terms, where .push would throw.
  • R3 - rest copy built in MLIRGen. [a, ...rest] = src gives rest a new array holding a copy of the slice (Array destructuring rest element aliases the source array (writes leak into it; rc crash on push) #477), built in MLIRGen (a new array of the slice's length and an element copy) instead of ts.ArrayView, so the ownership passes see a fresh array.
  • main's argv (replaces the plan's R2). argv is Ref<string> (C's char **) only. A main(argc, argv: string[]) is a compile error naming the Ref<string> form, under the JIT and ahead of time (--emit=exe/obj); a DLL has no main entry point and is not checked.
  • Amendment found by the default library gate (spec section 4, rc). The first version freed the data block outright when the header died ("the header holds its only reference"). The default library makes strings over an array's data block (<string><Opaque>Ref(buffer[0]) in convertNumber, convertInteger and the date formatters), and such a string takes a reference to the block, so under rc five default-library tests read freed memory ahead of time. Now under rc the data block keeps its own count: the header holds one reference (given when the header is made over the block, or when an empty array's first change allocates it), and the block and its elements are freed on the block's last reference, as on main. Under own the header still frees its block outright. A reallocating change still moves the block under such a string, as on main. Please overrule here if the intent was different.

Tests added

  • 00array_reference.ts - the 14 shapes of spec section 2 as asserts (compile, JIT, rc and none corpora).
  • 00array_rest_copy.ts (Array destructuring rest element aliases the source array (writes leak into it; rc crash on push) #477), 00array_zero_slots.ts, 00array_static_nested.ts, 00array_view.ts (updated: element counts per view).
  • 00array_data_string_alias.ts - a string made over an array's data block outlives the array (fails under rc, JIT, -O0 and -O3, without the amendment).
  • own/own_array_param_push.ts - a push through a borrowed parameter is the owner's (JIT, compile, no-counting); the pre-switch compiler (alpha88, gc) fails its assert.
  • lowering-errors/main_argv_string_array.ts - the argv error under --emit=jit, exe, obj, and no error under --emit=dll (linked from the build tree, -mm=none, so it does not depend on GC_LIB_PATH/TSLANG_LIB_PATH).

Gates

Gate Result
Windows Release suite (ctest -C Release -j 12 --timeout 300, DEFAULT_LIB_PATH = the rebuilt default library) 3734 / 3734 passed
Linux (WSL, this diff applied on origin/main 5e203fc, ninja release, ctest -j 8 --timeout 300) 3719 / 3719 passed (1 pre-existing disabled test). The first run failed only the argv DLL test (link needed GC_LIB_PATH), fixed in this PR
own corpus (--emit=llvm -mm=own --no-default-lib, every file of test/tester/tests) plain 454 / 593, --opt 452 / 593; main's baseline 453 / 593 and 451 / 593 (+1 is 00nullable_to_any, added on main after the baseline). No file goes from ok to failing. Changed first errors: 00types, callWithSpread (run-to-run hash in a name), 00var_bindings (still rejected, by the element-borrow rule instead)
own memory (measure.ps1, AOT -O3) flat - see below
default library rebuilt (release and debug, gc / rc / none) and its suite run under each model, release/debug x compile/JIT gc 159/159 x4; rc 158/159 x4; none 158/159 x4 - the one failure is weakref_basic, gc-only by design (#420)
test-compile-gc-defaultlib-collector (+ JIT twin) with the rebuilt library pass (fails against a library built before the switch, as expected)

Memory, MB (gc / rc / none / own):

program this PR own spec's last numbers
own_fresh_string 5.9 / 4.4 / 81.1 / 4.4 5.8 / 4.2 / 80.9 / 4.2
own_return_new 5.9 / 4.4 / 50.4 / 4.4 5.8 / 4.2 / 50.2 / 4.2
own_try_local 5.9 / 8.3 / 6.7 / 6.7 5.9 / 6.5 / 6.5 / 6.5
own_fresh_array 5.7 / 4.4 / 4.3 / 4.4 5.6 / 4.2 / 4.2 / 4.2
own_literals 5.7 / 4.4 / 4.3 / 4.4 5.6 / 4.1 / 4.1 / 4.1
own_array_param_push (new) 5.8 / 4.4 / 4.3 / 4.4 -
Shared leaf in two arrays, 20000 5.9 / 4.4 / 13.4 / 4.4 5.9 / 4.4 / 10.6 / 4.4
Shared leaf in two arrays, 200000 5.9 / 4.4 / 93.5 / 4.4 5.9 / 4.4 / 65.9 / 4.4
push through a parameter + spread copy, 200000 6.0 / 4.4 / 50.4 / 4.4 -

own and rc stay flat. none grows by the extra header per array (it frees nothing). own_try_local under rc is the known throw-path leak (a local is not released when the function throws, under every model): it now leaks a header and a data block per throw.

After merge

  • The default library must be rebuilt with this compiler for every model and both build types - the array ABI changed - including CI's default-library artifact. Installs built before this PR will not link with programs built after it.
  • Follow-up filed: Pushing onto an array inside a constant tuple literal crashes (realloc of static data) #479 - a push onto an array inside an unannotated constant tuple literal writes into constant global data (pre-existing; the old layout also pushed onto static data).

Closes #453
Closes #477

🤖 Generated with Claude Code

ASDAlexander77 and others added 18 commits October 3, 2026 22:48
An array value becomes one pointer to a heap header { data, length,
capacity } that never moves, so every alias sees push/pop/splice/length=.
Rollout in three PRs: a behaviour-free layout helper, the switch, growth.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Eight tasks in three PRs. Rulings folded into the spec: a null header
reads as an empty array (R1), main's string[] argv is made by a new op
(R2), the rest element is built in MLIRGen (R3).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
….cpp)

ArrayLayout reads and writes an array's data and length, and makes one.
No change to the generated IR: --emit=llvm identical for every suite test
under gc, rc and own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… ArrayLayout

The last places outside the helper that knew the { data, length } layout.
Also drops the llvmArrayType/loc locals Task 1 left unused.
No change to the generated IR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Closes #477

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An array value is one pointer to { data, length, capacity }. Assigning,
passing and storing an array copy the pointer, so a push, pop, splice or
length= through any name is seen through every other. Under rc the header
is the counted block; a null header reads as an empty array and gets a
header before a change through its slot.

Also: realloc and free of a null payload (an empty array's data) pass
null on instead of null minus the block word; `null` as an array is a
null header; .view() copies its slice; a constant array inside global
data gets a static, immortal header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`let g: C[];` left its slot unwritten. Reading an element of it now
compares the header pointer with null and branches on the result, and a
branch on an unwritten slot is undefined behaviour the optimizer took:
newWithSpread.ts faulted in every model. The slot starts null instead,
which reads as an empty array (ruling R1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`.view()` copies its slice into a new array, and the new array releases
its elements like any other; under rc it now takes a reference to each
one it copied, so releasing the view no longer frees elements the source
still holds. push, pop, shift, unshift, splice and length= write the
capacity with the length, keeping capacity = length until growth uses it.

The spec records the accepted limit of ruling R1 for a null slot passed
by value, and stale { data, length } comments describe the header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
view(from, to) includes `to`, so view(0, 1) is two elements: the class
case asserts it, and the string case asserts its one. Also reflows a
comment and drops two doubled blank lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	tslang/docs/superpowers/specs/2026-10-03-array-reference-design.md
#	tslang/include/TypeScript/LowerToLLVM/ArrayLayout.h
#	tslang/include/TypeScript/LowerToLLVM/CastLogicHelper.h
#	tslang/include/TypeScript/LowerToLLVM/LLVMCodeHelper.h
#	tslang/include/TypeScript/LowerToLLVM/OwnershipRoutineLogic.h
#	tslang/lib/TypeScript/LowerToLLVM.cpp
A string[] argv would read C's char ** as an array header. The main shape
check now accepts only Ref<string> and, for an array argv, says to use
Ref<string> and Deref(argv[i]). Spec section 5 and the plan are updated
(owner's ruling 2026-10-04; the ArrayFromCStrings op is dropped).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Linux (and wherever GC_LIB_PATH/TSLANG_LIB_PATH are unset) the DLL link
could not find gc or the runtime. -mm=none needs only the runtime, which
--tslang-lib-path points at in the build tree; the check is model-independent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A string made over an array's data block (`<string><Opaque>Ref(buffer[0])`,
the default library's convertNumber, convertInteger and date formatters)
takes a reference to the block. The release routine freed the block when
the header died, without reading its count, and a string over it then read
freed memory: five default-library tests failed ahead of time under rc.

The data block now gets a count of 1 when a header is made over it (or when
an empty array's first change allocates it), and the header's release drops
that reference; the elements are released and the block freed on its last
reference, as on main. Spec section 4 amended. 00array_data_string_alias
fails under rc (JIT, -O0 and -O3) without the change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only rc counts the data block (the header's reference plus any string made
over it). Under own the header is the block's only owner; the count test
there kept LLVM from removing a dead block, and own_try_local's known
throw-path leak grew from 6.7 to 8.3 MB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…h loop

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ASDAlexander77
ASDAlexander77 merged commit fbd7edc into main Oct 4, 2026
2 checks passed
@ASDAlexander77
ASDAlexander77 deleted the array-reference-switch branch October 4, 2026 09:46
ASDAlexander77 added a commit that referenced this pull request Oct 4, 2026
…484)

A tuple or record literal whose values are all constants is folded to a
ts.Constant, and each array in it was lowered to a header in constant
global data over the literal's constant data. push reallocated that
global; since #482 pop, shift, splice and `length =` wrote into it, and
the optimiser folded the reads back to the initializer, so they gave
silently wrong results.

Such a literal is now rebuilt with heap copies of its arrays wherever it
becomes a value that can be changed:
- a `const` of it gets storage and is widened to a tuple, as a `const`
  of a constant array is (processConstRef, adjustLocalVariableType,
  adjustGlobalVariableType);
- its cast to a tuple or a class rebuilds it from the literal's
  attributes (copyArraysOfConstTuple), nested tuple literals included;
- a constant array of such tuples is built element by element
  (castConstArrayToArray);
- inside a tuple literal built at run time, it is cast to a tuple;
- a record literal built in a slot (a field known only at run time, or
  boxed) sets its constant array fields as heap copies.

Under rc a module-level array or tuple literal is born at count 0 and
nothing took a count for the global, so the first local that read it
and let go freed it: `const ga = [6, 7]` read through a `const` twice
crashed (since #480 the length lives in the freed header). The global
now retains what its initializer builds (OwnedReturnConsumptionPass,
claimGlobalInitializers), as it already did for a call's result.

Closes #479

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant