Store compact cache indexes in bytecode instructions instead of raw
pointers to the executable cache vectors. This keeps the instruction
stream independent from heap addresses and removes pointer fixups when
materializing cached bytecode.
Resolve the mutable cache pointers at execution time from the current
Executable. Bytecode test expectations are updated for the smaller cache
operands and resulting instruction offsets.
Avoid decoding warm-cache script responses into full UTF-16 SourceCode
buffers when a bytecode cache sidecar is available. SourceCode now keeps
the original immutable source bytes and source encoding, then decodes
only when full source text or a Function.toString() range is requested.
Compute the bytecode cache source hash while streaming decoded code
points from the response bytes, so cache validation does not force an
intermediate UTF-8 string. Function and class source text metadata now
stores SourceCode ranges instead of views into a materialized buffer.
Store the environment serial number and catch-environment flag in
DeclarativeEnvironment::RareData instead of on every environment.
The serial is only used by global variable caches, and the catch flag
is only needed for catch clause environments, so shaped function
environments should not retain dedicated fields for either value.
Teach the asm global cache fast path to load the serial through rare
data and treat a missing sidecar as serial 0.
Move names, local flags, deletion state, binding lookup, dispose
state, and shape construction bookkeeping out of DeclarativeEnvironment
and into a lazily allocated sidecar. Shaped function environments keep
the shape pointer and value vector directly on the object, but do not
retain empty metadata containers after their shared shape has been
installed.
Keep the asm interpreter fast paths direct by loading local binding
flags from the sidecar only for unshaped environments or dynamic
bindings that come after a shared shape.
Let shaped declarative environments read static binding flags from their
EnvironmentShape instead of retaining a per-environment copy. The local
flag vector now mirrors the local name vector and only stores dynamic
bindings created after the shared shape.
Split EnvironmentShape binding descriptors into separate name and flag
vectors so the shared metadata is compact as well.
Store uninitialized declarative bindings as js_special_empty_value()
in the existing value slot instead of tracking initialization in a
separate bitmap. This removes one bitmap allocation from every
declarative environment while preserving the same TDZ checks.
Update the asm interpreter fast paths to test the value slot directly
and let initialization store the first non-empty value.
Cache EnvironmentShape for var environments created when default
parameter expressions force body vars into a separate environment. Reuse
that cache in CreateVariableEnvironment when the capacity matches the
active function metadata.
Expose active SharedFunctionInstanceData through VM so the bytecode
instruction can update the cache through a narrow API. Add coverage for
repeated calls with captured vars in this environment.
Cache EnvironmentShape on SharedFunctionInstanceData after the first
function environment creates its statically known bindings. Install the
cached shape before later calls create bindings so those environments
avoid building their own name vector and lookup map.
Let DeclarativeEnvironment keep a local overlay for eval-created
bindings, and keep the packed flag mirror per environment while the asm
interpreter still reads flags directly from the environment. Add runtime
coverage for repeated calls with an eval-created binding.
Store declarative environment values, names, flags, and initialization
state in separate containers instead of one Binding vector. This packs
per-binding flags into a byte and moves initialization state into an
AK::Bitmap. This reduces the per-environment storage we still need
before binding metadata can move into a shared EnvironmentShape.
Update the asm interpreter offsets and fast binding paths to load
values, flags, and initialization bits from the new storage. Keep
direct binding indices stable across deletion by clearing the slot
while removing the name from the lookup map. Cover that eval deletion
path in the local eval deoptimization tests.
The Annex B.3.4 spec change requires distinguishing
catch clause environments from other lexical environments
when checking for var/let conflicts in eval, which is now tracked
via a flag on DeclarativeEnvironment and propagated through the
CreateLexicalEnvironment bytecode instruction from the Rust codegen.
Replace the lazy per-shape OrderedHashMap cache for non-dictionary
shapes with a GC-allocated descriptor array. Store descriptors in hash
order for lookup while keeping an enum index so callers can still walk
properties in insertion order.
Keep dictionary shapes on the mutable OrderedHashMap path, and migrate
callers that enumerated Shape::property_table() to the new insertion
order iterator. Cap descriptor arrays to their compact u16 index range
and keep larger dictionary shapes on the mutable table path across
prototype transitions and prototype clones.
Add coverage for setting the prototype of a dictionary object with more
than 65536 named properties.
Keep basic block offsets as construction-only metadata rather than
storing them on every Executable. The validator now receives the offsets
through a transient Rust FFI span, and the bytecode dump rebuilds block
starts by scanning labels, terminators, and exception handler metadata.
Drop the table from the bytecode cache format and bump the format
version so old caches are rebuilt. This removes a field that was only
used by validation and bytecode dump paths.
Store source map locations as bytecode offset, line, and column.
Runtime consumers only emit the start line and column, so source end
positions and source text offsets do not need to be carried through
Executable source maps, bytecode cache serialization, or the Rust FFI.
Keep SourceCode's internal position cache able to track source text
offsets so callers can still translate source offsets to line and
column pairs when needed. Hash dump-bytecode IDs from the name, first
source position, and bytecode size instead of source slices that need
end offsets.
Bump the bytecode cache format version for the slimmer serialized
source map entry shape.
Avoid emitting consecutive source map entries when they carry the
same source range. The bytecode offset for the previous entry remains
valid for later PCs because source lookup now uses the largest source
map entry whose offset is not greater than the program counter.
This keeps stack traces stable while allowing statement-sized runs of
bytecode to share one source map entry.
Executable caches can retain weak pointers to shapes and prototypes
across collections. With incremental sweeping, a previous sweep may have
already freed the block behind one of those pointers by the time pruning
runs for weak containers.
Use the live HeapBlock registry before reading cached cells. This
matches the other weak containers updated for incremental sweeping.
Move weak container cleanup (remove_dead_cells) out of both
sweep_dead_cells() and start_incremental_sweep() to the place
where it is actually safe to inspect cell state: collect_garbage().
Previously, remove_dead_cells could access cells that had already
been swept and poisoned by ASAN, causing use-after-poison crashes
when a new GC triggered while an incremental sweep was in progress.
Report outline storage retained by scripts, environments, module
namespace objects, iterator helpers, property name iterators, argument
objects, and error tracebacks. These objects keep vectors and maps that
can grow independently from their GC cell sizes.
Report outline storage retained by bytecode executables, table
objects, object property iterator cache data, and shared function
instance data. This includes bytecode vectors, cache arrays, source
maps, class blueprint elements, and binding metadata.
Assert that inline Call frame-top arithmetic does not wrap after adding
the current interpreter stack top, and that raw native frames copy valid
caller lexical and variable environments.
Also check that the raw native ThrowCompletionOr variant index stays in
the expected 0-or-1 range before the asm fast path branches on it.
Assert that asm_helper_to_boolean returns only 0 or 1 before the asm
interpreter branches on the raw helper result. This documents the helper
ABI used by jump and logical-not fast paths without adding code to
release or distribution builds.
Add assertion-only checks for object shapes, binding storage, UTF-16
string backing storage, and iterator receiver shapes before the asm
interpreter relies on those pointers for follow-up loads.
These checks document invariants that are already required by the fast
paths, while still compiling away from release and distribution builds.
Replace assertion-only tag extraction around String helper return
values with assert_tag so release and distribution builds do not carry
code that only exists to feed assertions.
Asserting and non-asserting asmint generation both handle the updated
handlers.
Add debug-only assertions for typed-array cached data and property
iterator cache pointers before the asm fast paths dereference them. The
checks only consume values already loaded for normal execution, so
release and distribution builds still compile them out completely.
asmintgen successfully lowers this file for x86_64 with assertions
enabled.
Add debug-only assertions for value and frame contracts that cross the
asm/C++ boundary. Check resumed inline frames have executable bytecode,
lexical environments exist before boxing, and string builtin helpers
return string values for the fast paths that store their results.
asmintgen successfully lowers this file for x86_64 with assertions
enabled.
Add debug-only checks for relationships between fast-path metadata and
addressing bounds. These catch inconsistent inline-call slot counts,
argument padding, and flattened iterator cache lengths before the asm
path uses them for addressing.
asmintgen successfully lowers this file for x86_64 with assertions
enabled.
Add debug-only assertions around inline Call frame construction and
state restoration. The checks cover executable metadata, stack cursors,
realm/environment pointers, constant storage, raw native entry points,
and the VM state reloaded after helper calls.
asmintgen successfully lowers this file for x86_64 with assertions
enabled.
Add debug-only asmint assertions for cache and backing-store pointers
that fast paths dereference after metadata has selected a cache hit.
These checks catch stale property/global caches, broken environment
coordinates, and inconsistent indexed storage state without changing
release or distribution builds.
asmintgen successfully lowers this file for x86_64 with assertions
enabled.
Avoids a heap allocation per generator/async-generator resumption by
storing the pending completion value and type directly on the
GeneratorObject / AsyncGenerator, instead of allocating a separate
CompletionCell and passing it into the executable.
GetCompletionFields and SetCompletionType now read/write the fields on
the generator object directly.
The validator now bounds-checks the five enum-shaped field types
that appear in Bytecode.def: Completion::Type, IteratorHint,
EnvironmentMode, PutKind, and ArgumentsKind. The codegen
recognizes each by its .def type name and emits a u32 read plus a
range check against the corresponding variant count.
The variant counts ride across the FFI as new fields on
FFIValidatorBounds rather than being hardcoded on the Rust side,
so the Rust validator never has to know which variants the C++
enum currently defines. The C++ side computes each count as
`to_underlying(LastVariant) + 1` with a static_assert pinning the
expected value, so adding or removing a variant in any of these
enums fails the build until the validator is updated.
Until now the validator passed `u32::MAX` as the argument-region
upper bound because nothing on Executable tracked how many
argument slots a given bytecode buffer might reference. That left
the largest validation hole open: any flat operand index above
`registers + locals + constants` slid through the check.
The Rust assembler already walks every operand during phase 1 so
it can offset each one into the runtime's flat layout. This commit
piggybacks on that walk to record the highest `Operand::argument`
index touched and surfaces `(max + 1)` (or zero if no argument is
ever referenced) on `AssembledBytecode`. The value rides through
`FFIExecutableData` onto a new `Executable::number_of_arguments`
field, which `Validator.cpp` then feeds into `FFIValidatorBounds`.
The bound is now tight: every operand index in the encoded stream
is range-checked against the actual runtime array size, including
the argument region.
Pass 3 cross-checks the structural metadata stored alongside the
bytecode buffer on Executable against the offset set built during
Pass 1. Every basic block start offset must point at an instruction
boundary; exception handler start, end, and handler offsets must
either be at an instruction boundary or, for the inclusive-start /
exclusive-end pair, equal to the bytecode length; source map
entries must do the same.
Of these, the exception handler's handler_offset is the safety-
critical one for the disk-cache use case: a corrupted offset there
sends control flow into the middle of an instruction. The other
checks tighten the cache-load surface area and catch obvious file
corruption.
The metadata is plumbed across the FFI as a separate
FFIValidatorExtras struct so the validator entry point keeps the
single-call shape, with a flat-offset mirror struct for exception
handlers since the original carries no source data we need.
Pass 2 of the validator now runs a per-instruction check that walks
each opcode's fields and verifies every reference points somewhere
sensible. Operand indices, label addresses, identifier/string/
property-key/regex table indices, cache indices, and trailing
operand arrays are all bound-checked against the values the C++
side carries on the Executable. Fields whose bound depends on an
enum variant count or other type information not present in
Bytecode.def are left for a follow-up.
The codegen lives in build.rs and reuses the existing layout
machinery from the bytecode_def crate, so each opcode gets a match
arm whose body reads each field at its known byte offset and calls
the right hand-written validate_* helper. Variable-length
instructions cross-check the count field against m_length before
iterating the trailing array, which guards against an attacker
sneaking a count that walks off the end of the instruction.
Note that the encoded operand format is a flat u32 index into the
runtime [registers | locals | constants | arguments] array, since
Operand::offset_index_by zeroes the 3-bit type tag during assembly.
The validator therefore range-checks the flat index rather than
reading the type tag and dispatching per kind.
The argument-count upper bound isn't tracked on Executable yet, so
arguments remain effectively unbounded; tightening that bound is
left for a later commit.
Cache pointer fields are validated only when before_cache_fixup is
true, since after the fixup pass they hold real pointers and must
be left alone. NewFunction and NewClass have plain u32 fields for
shared-function-data and class-blueprint indices; those are
recognized by name in the codegen so the indices still get
range-checked.
The error category enum is renumbered to drop the per-operand-kind
codes, since at the bytecode level we no longer differentiate.
The plan is to start caching compiled JS bytecode on disk. Before
loading anything from a cache we need confidence that the bytes are
structurally well-formed, since a corrupted or tampered-with cache
file could otherwise hand the interpreter an out-of-bounds jump or a
constant-pool index that points past the end of the table.
This commit lays down the scaffolding for that validator. The walker
lives in Rust (Libraries/LibJS/Rust/src/bytecode/validator.rs) so
that it can share the existing Bytecode.def-driven layout machinery
with the encoder. C++ calls into it through cbindgen, the same way
the rest of the Rust pipeline is wired up.
For now, the validator only does Pass 1: walk the byte stream,
verify each instruction is 8-byte aligned, the opcode byte is in
range, and the reported length keeps us inside the buffer. The
length lookup is generated from Bytecode.def so fixed-length and
variable-length instructions stay in sync with the rest of the
codegen automatically. Per-field bounds checks (operands, labels,
table indices, cache indices) and structural extras (basic block
offsets, exception handlers, source map) come in follow-up commits.
The validator runs after every successful compilation in debug and
sanitizer builds, gated on !NDEBUG || HAS_ADDRESS_SANITIZER, so we
get an extra sanity check on every executable the encoder produces
without paying for it in release builds. Failure trips a
VERIFY_NOT_REACHED with the offset, opcode, and error category
logged via dbgln().
Track whether a function needs environment-backed this resolution
separately from whether it needs to allocate its own function
environment. Arrow functions that only capture lexical this can now
resolve through the outer environment without allocating an empty
function environment for every call.
Keep the asm Call path conservative by routing functions that still need
lexical-this resolution through the C++ inline-call helper, so the call
receiver is not cached as the arrow function's this value.
Microbenchmark:
function makeLexicalThisArrow() {
return () => this.value;
}
let object = { value: 1, makeLexicalThisArrow };
let fn = object.makeLexicalThisArrow();
for (let i = 0; i < 20_000_000; ++i)
fn();
Measured with the same Release build toggling this patch:
baseline: 1069.2 ms mean over 12 runs
optimized: 501.2 ms mean over 12 runs
speedup: 2.13 times faster
asmint.asm no longer references any positional temp register name --
every handler and macro declares its temporaries by name with `temp` /
`ftemp` and lets the register allocator place them. Migrate the last
two macros holding out:
* dispatch_current uses a macro-local `opcode` temp for the load8 +
indirect jmp.
* pop_inline_frame_and_resume names its return-pc, dst-index, value-
address, vm-pointer, and executable temps explicitly.
With nothing left referring to the positional aliases, drop the
tN / ftN -> physical-register fallback from registers::resolve_register
and update the DSL reference comments at the top of asmint.asm and in
main.rs to describe the named-temp model. The two pre-existing codegen
tests that probed the old positional behavior get rewritten to use the
post-allocation physical-register names directly, since that is now
the actual contract of resolve_op.
The Call handler is the asm interpreter's hot path for inline JS-to-JS
dispatch, and it carries roughly fifty distinct named values across
its body (callee object, packed metadata word, formal/passed/total
counts, two stack-side pointers, the argument-copy cursor, the
script-or-module pair-load scratch, the native return value and
variant tag, ...).
The native-exception path adds a small `mov helper_arg, native_return`
bridge between call_raw_native (output pinned to rax by
fixed_operands) and call_helper (input pinned to rcx by
fixed_operands), since the two ABIs land their values in different
registers and the explicit mov is the cleanest way to express the
hand-off.
Convert the for-in iteration fast path to named temps. The handler
threads ~25 distinct values through its body (the iterator value,
its tag, the unboxed iterator, the fast-path discriminator, the
property cache pointer, the cached and current shapes, the receiver,
the dictionary-generation pair, the indexed-count and next-indexed
counters, the named-index, the named-key data pointer, and a few
scratch slots), so the migrated form is much easier to read than the
positional spaghetti where t0..t8 each meant five different things in
different basic blocks.
Convert the array / typed-array load fast paths in GetByValue, the
mirror of PutByValue. Roughly the same shape: ~18 named GPR temps
and a slot_dbl FPR temp covering the loaded value, the bytes-or-int
raw integer view, the address scratch, and the negative-zero compare.
The .ta_float32 / .ta_float64 paths become readable: where the
positional code had "Exclude negative zero early (t1 gets clobbered
by double_to_int32)" because t1 had to do double duty, the migrated
form names slot, slot_dbl, neg_zero, and raw separately and the
allocator lays them out without the implicit clobber dance.
Convert the array / typed-array store fast paths in PutByValue --
the largest single handler so far at 150+ positional t-uses -- to
declare its 19 GPR temps and 1 FPR temp by name (kind, base, prop,
index, obj, flags, storage_kind, size, elements, src, capacity_addr,
capacity, slot, empty_tag, kind_byte, addr, src_int32, max, result,
src_dbl, plus base_tag/prop_tag scratches reused across paths).
The migrated body removes the "save in t0 before load_operand
clobbers it" / "compute store address before check_is_double clobbers
t4" maneuvers that the positional code needed -- the allocator
handles those constraints automatically.
Convert the load_primitive_string_utf16_code_unit macro to take its
inputs (string pointer, index) and output (code_unit) as explicit
parameters, plus migrate CallBuiltinStringFromCharCode,
CallBuiltinStringPrototypeCharCodeAt, and
CallBuiltinStringPrototypeCharAt.
The macro previously hard-coded inputs to t2/t4 and output to t0,
which forced callers to remember those slot assignments and to wrap
the call_helper interactions around them. The migrated form -- where
the caller names what it wants in each slot -- removes another
"clobbers t3, t5" comment.
Convert the global variable IC and the array-length fast path to
named temps. These three handlers are large -- GetGlobal and
SetGlobal carry ~20 named values across each path (realm pointer,
global object, declarative environment, cache pointer, two serial
numbers, the shape, the cached shape, the dictionary generations,
the property offset, the named-properties pointer, the loaded value
and tag, plus environment-binding state) -- and the migration makes
those values easy to read instead of having to remember which slot
each piece of state was occupying.
GetLength's "magical" int-to-double widening picks up a named
temp for the sign-bit check and the result, which removes another
"NB: load_operand clobbers t0" comment.
Convert the inline-cache fast paths for property load/store to named
temps. Both handlers carry a substantial amount of state across the
ic-hit check (the boxed base value, its tag, the unboxed Object*,
the shape, the PLC pointer, the cached shape and prototype, the
property offset, the current and cached dictionary generations, the
named-properties pointer, and the loaded value), so giving each
piece an explicit name removes a real readability burden. The IC-miss
path uses the new 3-operand call_interp form.
Notably, PutById's old code carried a "save property offset in t4
before load_operand clobbers t0 (rax)" maneuver -- that is exactly
the kind of cross-cut the allocator handles automatically once the
offset is named. The migrated handler just keeps using `prop_offset`
across the load_operand and the store, no scratch shuffle needed.
Convert the validate_callee_builtin macro and the Math.abs / floor /
ceil / sqrt / exp / and ResolveThisBinding handlers to named DSL
temporaries. Math.exp uses the 3-operand call_helper form; the
allocator pins its `arg` to t1 (rcx) and `result` to t0 (rax)
automatically.
Convert the Div and Mod handlers to named DSL temporaries. Mod is the
first migrated handler that uses divmod, with its quot/rem pinned to
rax/rdx by fixed_operands and its dividend/divisor kept off both by
the operand-vs-implicit-output interference rule.
Convert the walk_env_chain macro to take its outputs (target_env,
bind_index) as explicit parameters, then migrate every handler that
walks the environment chain: GetBinding, GetInitializedBinding,
InitializeLexicalBinding, SetLexicalBinding, and
GetCalleeAndThisFromEnvironment.
The macro's internal scratch (the EnvironmentCoordinate address, the
hops counter, the invalid-coord sentinel, and the screwed-by-eval
flag) is now declared as macro-local temps that the allocator places
without touching whatever the caller has live. Each handler picks
its own names for env / idx / binding / value, instead of remembering
that the macro happens to leave them in t3 and t2.
Convert coerce_to_int32s and bitwise_op to take operand temps as
explicit parameters, then migrate UnaryPlus.
The bitwise_op macro is the actual body of BitwiseAnd, BitwiseOr, and
BitwiseXor (which just call into it), so a single macro migration
covers all three handlers. UnaryPlus is migrated alongside since it
shares the same coerce-to-int32 conceptual path.
Convert the equality dispatch chain to take operand temps as explicit
parameters: equality_same_tag, double_equality_compare,
strict_equality_core, and loose_equality_core. Then migrate the
StrictlyEquals, StrictlyInequals, LooselyEquals, LooselyInequals,
JumpStrictlyEquals, JumpStrictlyInequals, JumpLooselyEquals, and
JumpLooselyInequals handlers.
These macros are tightly coupled -- strict_equality_core and
loose_equality_core both invoke equality_same_tag and reach
double_equality_compare via .double_compare, a label that crosses
macro boundaries. The existing label-uniquification rule (only
self-contained labels are renamed) keeps that contract working
without further plumbing.
Convert the heavy comparison and arithmetic macros to take their
operand temps as explicit parameters, then migrate every handler that
uses them: coerce_to_doubles, numeric_compare, numeric_compare_coerce,
boolean_result_epilogue, jump_binary_epilogue, plus the Add, Sub, Mul,
LessThan, LessThanEquals, GreaterThan, GreaterThanEquals,
JumpLessThan, JumpGreaterThan, JumpLessThanEquals, and
JumpGreaterThanEquals handlers.
Convert check_is_double, check_both_double, and box_double_or_int32 to
declare their internal scratch as macro-local `temp` decls instead of
hardcoding t3/t4. Each invocation now gets a freshly uniquified temp
that the allocator places independently of any other live values in
the caller, so adding a `temp` declaration around a sequence that
calls into one of these macros no longer needs to know which
positional slot the macro happened to be using.
box_double_or_int32 in particular is on the hot arithmetic path
(Add/Sub/Mul/PostfixIncrement/etc.), so this opens the door to
migrating those handlers without conflicting with the macro's
internal scratch use.
Convert another batch of macro-free handlers to named DSL temporaries.
The shift handlers in particular are nice exercises of the allocator's
fixed-operand support: x86's `shl`/`shr`/`sar` need the count register
to be rcx, and the allocator pins the named `count` temp accordingly
without the user having to remember it.
Convert another batch of straightforward handlers to named DSL
temporaries: Increment, Decrement, Not, GetLexicalEnvironment, Return,
and End. The migration uncovers a pleasing property of the new
pipeline: when a migrated handler invokes an un-migrated macro
(pop_inline_frame_and_resume here, which still uses positional t2/t3
/t4 internally), the macro body's positional uses appear as physical-
register defs in the flat instruction list and the allocator's
liveness analysis avoids placing live named temps in those registers.
Convert the three boolean-conditional jump handlers to named temps,
exercising the 3-operand call_helper form so the input value and
truthy result no longer have to be pinned to t1/t0 by hand. The
allocator places `condition` in rcx (matching call_helper's input
convention) and `truthy` in rax (matching its output) automatically.
Convert Jump, JumpNullish, and JumpUndefined to declare named
temporaries instead of pinning values to t0..t2 directly. These three
handlers are macro-free and call-free, so the migration is mechanical
and the allocator picks register assignments equivalent in cost to
the hand-written positional code.
The other Jump* handlers (JumpIf/JumpTrue/JumpFalse) drive
call_helper, which still needs the value to be in t1 (rcx) on x86 by
the existing register convention. Migrating those needs DSL syntax to
attach call_helper's input/output to named temps; until that lands
they stay on the positional form.
These three trivial data-movement handlers are a useful first
migration target -- they exercise the new `temp` declaration syntax
and the register allocator end-to-end without interacting with macros
or fixed-operand constraints. The allocator picks t1-equivalent
registers for the first temp (which is the smallest live range that
must survive load_operand's rax/x9 scratch use) and reuses the slot
across non-overlapping live ranges where possible.
Carry full source positions through the Rust bytecode source map so
stack traces and other bytecode-backed source lookups can use them
directly.
This keeps exception-heavy paths from reconstructing line and column
information through SourceCode::range_from_offsets(), which can spend a
lot of time building SourceCode's position cache on first use.
We're trading some space for time here, but I believe it's worth it at
this tag, as this saves ~250ms of main thread time while loading
https://x.com/ on my Linux machine. :^)
Reading the stored Position out of the source map directly also exposed
two things masked by the old range_from_offsets() path: a latent
off-by-one in Lexer::new_at_offset() (its consume() bumped line_column
past the character at offset; only synthesize_binding_pattern() hit it),
and a (1,1) fallback in range_from_offsets() that fired whenever the
queried range reached EOF. Fix the lexer, then rebaseline both the
bytecode dump tests (no more spurious "1:1") and the destructuring AST
tests (binding-pattern identifiers now report their real columns).
GetCalleeAndThisFromEnvironment treated a binding as initialized when
its value slot was not <empty>. Declarative bindings do not encode TDZ
in that slot, though: uninitialized bindings keep a separate initialized
flag and their value starts as undefined.
That let the first slow-path TDZ failure populate the environment cache,
then a second call at the same site reused the cached coordinate and
turned the required ReferenceError into a TypeError from calling
undefined.
Check Binding.initialized in the asm fast path instead and cover the
cached second-hit case with a regression test.
The asm interpreter already inlines ECMAScript calls, but builtin calls
still went through the generic C++ Call slow path even when the callee
was a plain native function pointer. That added an avoidable boundary
around hot builtin calls and kept asm from taking full advantage of the
new RawNativeFunction representation.
Teach the asm Call handler to recognize RawNativeFunction, allocate the
callee frame on the interpreter stack, copy the call-site arguments,
and jump straight to the stored C++ entry point.
NativeJavaScriptBackedFunction and other non-raw callees keep falling
through to the existing C++ slow path unchanged.
Place Realm's cached declarative environment next to its global object
so the asm global access fast paths can fetch the two pointers with a
paired load. These handlers never use the intervening GlobalEnvironment
pointer directly.
Mirror Executable's constants size and data pointer in adjacent fields
so the asm Call fast path can pair-load them together. The underlying
Vector layout keeps size and data apart, so a small cached raw span
lets the hot constant-copy loop fetch both pieces of metadata at once.
Load PropertyNameIterator's indexed-property count and next index
together when stepping the fast path. Keeping the paired count live
into the named-property case also avoids reloading it before computing
the flattened index.
Load PropertyNameIterator's cached property cache and shape snapshot
together before validating the receiver shape. The two fields already
sit adjacent in the object layout, so the fast path can fetch both
without any extra reshuffling.
Load EnvironmentCoordinate::hops and ::index together in the asm
environment-walk helper. The pair-load keeps the DSL explicit about
which two fields travel together and removes another scalar metadata
fetch from the fast path.
Load the cached property offset and dictionary generation with paired
loads in the property inline-cache fast paths. AsmIntGen now verifies
these reads against the actual cache layout, so the DSL keeps both
fields named and self-documenting.
Load the cached shape and prototype pointer together in the property
inline-cache fast paths that already read both. This keeps the
cache-entry metadata fetches aligned with the DSL's paired-load model
without changing the surrounding control flow.
Load the inline frame's return pc and destination register at once when
Return or End resumes an asm-managed caller. This keeps the unwind
metadata with the helper that consumes it and removes a separate scalar
load from both handlers.
The asm Call fast path was still reloading the executable pointer while
building the inline callee frame, even though it had already loaded the
same pointer while validating the call target.
Carry that executable pointer through frame setup and reload the passed
argument count from the call bytecode instead of the fresh frame header.
This trims a couple more loads from the hot path.
Pack the asm Call fast path metadata next to the executable pointer
so the interpreter can fetch both values with one paired load. This
removes several dependent shared-data loads from the hot path.
Keep the executable pointer and packed metadata in separate registers
through this binding so the fast path can still use the paired-load
layout after any non-strict this adjustment.
Lower the packed metadata flag checks correctly on x86_64 as well.
Those bits now live above bit 31, so the generator uses bt for single-
bit high masks and covers that path with a unit test.
Add a runtime test that exercises both object and global this binding
through the asm Call fast path.
Executable already caches the combined registers, locals, and constants
count that the asm Call fast path needs for inline frame allocation.
Use that precomputed total instead of rebuilding it from the registers
count and constants vector size in the hot path.
The asm Call fast path already checks SharedFunctionInstanceData's
cached can_inline_call bit before touching the executable pointer.
That cache is only true for ordinary functions with compiled bytecode,
so the extra executable null check is redundant work in the hot path.
The asm Call fast path reads InterpreterStack::m_top and m_limit
back-to-back while checking whether the inline callee frame fits.
Those fields are adjacent, so we can load them together with one
paired load and keep the stack-size check otherwise unchanged.
Teach the asm Call fast path to use paired stores for the fixed
ExecutionContext header writes and for the caller linkage fields.
This also initializes the five reserved Value slots directly instead
of looping over them as part of the general register clear path.
That keeps the hot frame setup work closer to the actual data layout:
reserved registers are seeded with a couple of fixed stores, while the
remaining register and local slots are cleared in wider chunks.
On x86_64, keep the new explicit-offset formatting on store_pair*
and load_pair* without changing ordinary [base, index, scale]
operands into base-plus-index-plus-offset addresses. Add unit
tests covering both the paired zero-offset form and the preserved
scaled-index lowering.
Use the new paired-load DSL operations in the inline Call path for the
adjacent environment, ScriptOrModule, caller metadata, and callee-entry
loads. The flow stays the same, but the hot call setup now needs fewer
scalar memory operations on aarch64.
Handle inline-eligible JS-to-JS Call directly in asmint.asm instead
of routing the whole operation through AsmInterpreter.cpp.
The asm handler now validates the callee, binds `this` for the
non-allocating cases, reserves the callee InterpreterStack frame,
populates the ExecutionContext header and Value tail, and enters the
callee bytecode at pc 0.
Keep the cases that need NewFunctionEnvironment() or sloppy `this`
boxing on a narrow helper that still builds an inline frame. This
preserves the existing inline-call semantics for promise-job ordering,
receiver binding, and sloppy global-this handling while keeping the
common path in assembly.
Add regression coverage for closure-capturing callees, sloppy
primitive receivers, and sloppy undefined receivers.
Emit the ExecutionContext, function-object, executable, and realm
offsets that the asm Call path needs to inspect and initialize
directly when building inline frames.
Handle Return and End entirely in AsmInt when leaving an inline frame.
The handlers now restore the caller, update the interpreter stack
bookkeeping directly, and bump the execution generation without
bouncing through AsmInterpreter.cpp.
Add WeakRef tests that exercise both inline Return and inline End
so this path stays covered.
Store whether a function can participate in JS-to-JS inline calls on
SharedFunctionInstanceData instead of recomputing the function kind,
class-constructor bit, and bytecode availability at each fast-path
call site.
Keep JS-to-JS inline calls out of m_execution_context_stack and walk
the active stack from the running execution context instead. Base
pushes now record the previous running context so duplicate
TemporaryExecutionContext pushes and host re-entry still restore
correctly.
This keeps the fast JS-to-JS path off the vector without losing GC
root collection, stack traces, or helpers that need to inspect the
active execution context chain.
The bytecode interpreter only needed the running execution context,
but still threaded a separate Interpreter object through both the C++
and asm entry points. Move that state and the bytecode execution
helpers onto VM instead, and teach the asm generator and slow paths to
use VM directly.
Specialize only the fixed unary case in the bytecode generator and let
all other argument counts keep using the generic Call instruction. This
keeps the builtin bytecode simple while still covering the common fast
path.
The asm interpreter handles int32 inputs directly, applies the ToUint16
mask in-place, and reuses the VM's cached ASCII single-character
strings when the result is 7-bit representable. Non-ASCII single code
unit results stay on the dedicated builtin path via a small helper, and
the dedicated slow path still handles the generic cases.
Tag String.prototype.charAt as a builtin and emit a dedicated
bytecode instruction for non-computed calls.
The asm interpreter can then stay on the fast path when the
receiver is a primitive string with resident UTF-16 data and the
selected code unit is ASCII. In that case we can return the VM's
cached empty or single-character ASCII string directly.
Teach builtin call specialization to recognize non-computed
member calls to charCodeAt() and emit a dedicated builtin opcode.
Mark String.prototype.charCodeAt with that builtin tag, then add
an asm interpreter fast path for primitive-string receivers whose
UTF-16 data is already resident.
The asm path handles both ASCII-backed and UTF-16-backed resident
strings, returns NaN for out-of-bounds Int32 indices, and falls
back to the generic builtin call path for everything else. This
keeps the optimistic case in asm while preserving the ordinary
method call semantics when charCodeAt has been replaced or when
string resolution would be required.
Replace the generic CallBuiltin instruction with one opcode per
supported builtin call and make those instructions fixed-size by
arity. This removes the builtin dispatch sled in the asm
interpreter, gives each builtin a dedicated slow-path entry point,
and lets bytecode generation encode the callee shape directly.
Keep the existing handwritten asm fast paths for the Math builtins
that already benefit from them, while routing the other builtin
opcodes through their own C++ execute implementations. Build the
new opcode directly in Rust codegen, and keep the generic call
fallback when the original builtin function has been replaced.
Cache the flattened enumerable key snapshot for each `for..in` site and
reuse a `PropertyNameIterator` when the receiver shape, dictionary
generation, indexed storage kind and length, prototype chain
validity, and magical-length state still match.
Handle packed indexed receivers as well as plain named-property
objects. Teach `ObjectPropertyIteratorNext` in `asmint.asm` to return
cached property values directly and to fall back to the slow iterator
logic when any guard fails.
Treat arrays' hidden non-enumerable `length` property as a visited
name for for-in shadowing, and include the receiver's magical-length
state in the cache key so arrays and plain objects do not share
snapshots.
Add `test-js` and `test-js-bytecode` coverage for mixed numeric and
named keys, packed receiver transitions, re-entry, iterator reuse, GC
retention, array length shadowing, and same-site cache reuse.
Teach the asm PutByValue path to materialize in-bounds holey array
elements directly when the receiver is a normal extensible Array with
the default prototype chain and no indexed interference. This avoids
bouncing through generic property setting while preserving the lazy
holey length model.
Keep the fast path narrow so inherited setters, inherited non-writable
properties, and non-extensible arrays still fall back to the generic
semantics. Add regression coverage for those cases alongside the large
holey array stress tests.
Treat setting a large array length as a logical length change instead of
forcing dictionary indexed storage or materializing every hole up front.
This keeps dense fills on Array(length) on the holey indexed path and
only falls back to sparse storage when later writes actually create a
large realized gap.
The asm indexed get/put fast paths assumed holey arrays always had a
materialized backing store. Guard those paths with a capacity check so
lazy holey arrays fall back safely until an index has been realized.
Add regression coverage for very large holey arrays and for densely
filling a large holey array after pre-sizing it with Array(length).
This better describes what the method returns and avoids the possible
confusion caused by the mismatch in behavior between
`Value::is_array()` and `Value::as_array()`.
In b2d9fd3352, the root cause of the crash
was somewhat misdiagnosed, particularly around what place in code an
allocation could occur while constants data was uninitialized.
But more importantly, we can do better than the solution in that commit.
Instead of initializing constants with default values and then
overwriting them afterwards, simply initialize them with their actual
values directly when constructing the execution context.
This effectivly reverts commit b2d9fd3352.
The additional data being passed will be used in an upcoming commit.
Allows splitting the churn of modified function signatures from the
logically meaningful code change.
No behavior change.
Switch LibJS `RegExp` over to the Rust-backed `ECMAScriptRegex` APIs.
Route `new RegExp()`, regex literals, and the RegExp builtins through
the new compile and exec APIs, and stop re-validating patterns with the
deleted C++ parser on the way in. Preserve the observable error
behavior by carrying structured compile errors and backtracking-limit
failures across the FFI boundary. Cache compiled regex state and named
capture metadata on `RegExpObject` in the new representation.
Use the new API surface to simplify and speed up the builtin paths too:
share `exec_internal`, cache compiled regex pointers, keep the legacy
RegExp statics lazy, run global replace through batch `find_all`, and
optimize replace, test, split, and String helper paths. Add regression
tests for those JavaScript-visible paths.
Add `ECMAScriptRegex`, LibRegex's C++ facade for ECMAScript regexes.
The facade owns compilation, execution, captures, named groups, and
error translation for the Rust backend, which lets callers stop
depending on the legacy parser and matcher types directly. Use it in the
remaining non-LibJS callers: URLPattern, HTML input pattern handling,
and the places in LibHTTP that only needed token validation.
Where a full regex engine was unnecessary, replace those call sites with
direct character checks. Also update focused LibURL, LibHTTP, and WPT
coverage for the migrated callers and corrected surrogate handling.
Dictionary shapes are mutable (properties added/removed in-place via
add_property_without_transition), so sharing them between objects via
the NewObject premade shape cache is unsafe.
When a large object literal (>64 properties) is created repeatedly in
a loop, the first execution transitions to a dictionary shape, which
CacheObjectShape then caches. Subsequent iterations create new objects
all pointing to the same dictionary shape. If any of these objects adds
a new property, it mutates the shared shape in-place, increasing its
property_count, but only grows its own named property storage. Other
objects sharing the shape are left with undersized storage, leading to
a heap-buffer-overflow when the GC visits their edges.
Fix this by not caching dictionary shapes. This means object literals
with >64 properties won't get the premade-shape fast path, but such
literals are uncommon.
Store yield_continuation and yield_is_await directly in
ExecutionContext instead of allocating a GeneratorResult GC cell.
This removes a heap allocation per yield/await and fixes a latent
bug where continuation addresses stored as doubles could lose
precision.
We specialize `Optional<T>` for value types that inherently support some
kind of "empty" value or whose value range allow for a unlikely to be
useful sentinel value that can mean "empty", instead of the boolean flag
a regular Optional<T> needs to store. Because of padding, this often
means saving 4 to 8 bytes per instance.
By extending the new `SentinelOptional<T, Traits>`, these
specializations are significantly simplified to just having to define
what the sentinel value is, and how to identify a sentinel value.
Add a metadata header showing register count, block count, local
variable names, and the constants table. Resolve jump targets to
block labels (e.g. "block1") instead of raw hex addresses, and add
visual separation between basic blocks.
Make identifier and property key formatting more concise by using
backtick quoting and showing base_identifier as a trailing
parenthetical hint that joins the base and property names.
Generate a stable name for each executable by hashing the source
text it covers (stable across codegen changes). Named functions
show as "foo$9beb91ec", anonymous ones as "$43362f3f". Also show
the source filename, line, and column.
ScopedOperand was a ref-counted wrapper around Operand used by the
C++ bytecode Generator for register lifetime tracking. Now that the
Generator is gone, it's just a pointless indirection.
Update the bytecode def code generator to emit Operand directly
instead of ScopedOperand in variable-argument op constructors, and
delete ScopedOperand.h.
Clean up leftover references to the removed C++ pipeline:
- Remove stale forward declarations from Forward.h (ASTNode,
Parser, Program, FunctionNode, ScopeNode, etc.)
- Delete unused FunctionParsingInsights.h
- Remove dead get_builtin(MemberExpression const&) declaration
from Builtins.h
- Update stale comments referencing ASTCodegen.cpp and
generate_bytecode()
Delete AST.cpp, AST.h, ASTDump.cpp, ScopeRecord.h, and the dead
get_builtin(MemberExpression const&) from Builtins.cpp.
Extract ImportEntry and ExportEntry into a new ModuleEntry.h,
since they are data types used by the module system, not AST
node types.
Inline ModuleRequest's sorting constructor and
SourceRange::filename().
Remove the dead annex_b_function_declarations field from
EvalDeclarationData, which was only populated by the C++ parser.
