Produce JS-visible string results as UTF-16 at their source, including
numeric formatting, BigInt and BigFraction formatting, URI encoding,
console formatting, parser errors, regular expression errors, Intl and
Temporal records, LibUnicode locale boundaries, and LibWeb bindings.
Handle fractional radix formatting through the UTF-16 builder view.
The interpreter's fast path for PutByValue on a typed array treated an
out-of-bounds index as a silent no-op and returned without touching the
value. That is observably wrong: TypedArraySetElement evaluates
ToNumber(value) before checking the index, so a value with a valueOf
side effect must still have that side effect run even when the store is
ultimately discarded.
Fall back to the slow path on an out-of-bounds or otherwise invalid
index instead of reporting success. The slow path runs the full
TypedArraySetElement algorithm, which performs the coercion and then
discards the write. Direct assignment now matches Reflect.set, which
already went through the slow path.
Fixes the staging/sm typed array out-of-bounds ToNumber test262 case
and adds a test-js regression covering direct assignment, Reflect.set,
and Reflect.defineProperty.
Move the execution context program counter update from ASM_TRY() to the
generated slow-path call boundary. Slow paths still enter C++ with the
current bytecode offset visible to stack and source location code, while
ASM_TRY() only handles completion unwrapping and exception dispatch.
Have generated AsmInt calls pass the current instruction pointer as a
third argument to slow-path handlers. This lets the C++ handlers use a
typed Op pointer directly instead of refetching bytecode from the VM and
recomputing the instruction address from the program counter.
Remove the optional slow path hit counters from AsmSlowPaths.cpp. This
also drops the registration call from the AsmInt entry path, leaving
slow paths focused on executing the out-of-line instruction behavior.
Remove the empty AsmInterpreter wrapper and the VM::run_bytecode()
trampoline now that the bytecode interpreter only enters AsmInt. Move
the stack-limit check and generated assembly entry call into
run_executable(), then drop the stale wrapper source file and includes.
Remove VM helpers that became unused after bytecode execution stopped
using the generic interpreter path. The AsmInt entry path now owns these
transitions directly.
Move the remaining simple SetLexicalEnvironment, IsCallable and
LeavePrivateEnvironment opcodes into the AsmInt DSL. These handlers do
not need C++ slow-path support.
Move the C++ slow paths used by AsmInt into their own translation unit.
This leaves Interpreter.cpp focused on VM entry and bytecode metadata
helpers instead of carrying the slow-path implementation body.
Move the remaining bytecode instruction implementations out of
execute_impl() and into AsmInt slow paths. Remove the execute_impl()
bodies once their only caller is gone, leaving instruction classes as
bytecode data containers.
Remove the generic fallback dispatch once every bytecode opcode has a
real AsmInt handler. Invalid dispatch table entries still route through
the fallback function as a defensive trap.
Move property access, iterator, object property iterator, import, class
and argument-array call opcodes out of the generic fallback path. Keep
the semantic work in C++ slow paths and dispatch to them from AsmInt.
Move the remaining control-flow, conversion, creation, delete, binding,
private-name and environment-related fallback handlers into AsmInt. This
keeps the generic fallback path shrinking while leaving complex behavior
in C++ slow paths.
Move simple fallback handlers into the AsmInt DSL or dedicated slow-path
calls. This covers straightforward allocation, environment setup,
argument creation, completion state, template object, async iterator and
function allocation opcodes.
Remove the C++ bytecode interpreter dispatch loop now that AsmInt is the
only bytecode execution engine. Keep the existing AsmInt fallback path
for instructions that have not yet been moved into assembly or C++ slow
path handlers.
Enable the asm interpreter on Windows for both x86_64 and ARM64. The
x86_64 backend now emits COFF assembly using the Win64 ABI. It handles
argument registers, non-volatile register saves, shadow space, SEH
unwind directives, and raw-native sret lowering. Its epilogue is left
as normal x64 instructions instead of ARM64-style SEH epilogue
directives, which older ClangCL assemblers reject.
The AArch64 backend now emits Windows ARM64 COFF assembly as well,
including COFF relocations, .rdata dispatch tables, SEH unwind metadata,
frame sizing, handler alignment rules, and raw-native sret lowering.
CMake selects COFF for Windows asmint output and enables generation for
both Windows architectures.
AsmIntGen coverage checks the Windows x64 epilogue output and the ARM64
COFF unwind output. test-js and test262 have no regressions with asmint
enabled compared with the C++ interpreter.
Only drain the standalone Promise job queue when unwinding the outermost
bytecode execution. Draining after every nested run_executable() return
let sibling Promise reactions run before current-job reactions.
Plain async function returns were relying on that extra drain because
return values were compiled as implicit awaits. Resolve completed async
functions through the Promise resolving function instead, while keeping
the async generator return-await path.
Add Promise ordering coverage and update the asm inline-call regression
to check the pre-drain state.
Resolve dynamic identifier assignment targets before evaluating the
right hand side, then store through that saved environment record. This
matches the ECMA-262 ordering for simple assignment and var initializers
when a with binding is deleted or direct eval creates a nearer var.
Add runtime coverage for with and direct-eval cases and refresh bytecode
expectations for dynamic assignments that now snapshot the binding
before storing.
Match ECMA-262's SuperCall evaluation order by resolving the super
constructor before evaluating the argument list. This preserves the
constructor across argument-side prototype mutations and still lets
abrupt argument evaluation happen before the constructor check.
Add LibJS coverage and update bytecode baselines for the new saved
super-constructor operand. The test262 superCallOrder staging test now
passes.
Sync yield* must suspend with the delegated IteratorResult object itself
on non-terminal steps. We were reading value and yielding that value,
causing Generator.prototype.next() to create a fresh result object.
Add a bytecode op for this path so ordinary yield keeps wrapping its
value, and teach generator resume to return the prebuilt result object.
This also avoids touching value before the delegated iterator is done.
Add regression coverage and update the yield* bytecode baseline.
Use a runtime SetFunctionName bytecode operation when object literal
property keys are not known until evaluation. This lets anonymous
function and class expressions, methods, and accessors receive names
from numeric, computed, and Symbol property keys.
Store inferred ECMAScript function names on each function object instead
of mutating shared function data. That keeps repeated evaluations with
different computed keys from leaking names across closures, while still
using the per-instance name for stack metadata.
Add regression coverage for computed object property names, repeated
computed-key evaluations, and preserving unnamed functions that are only
referenced by a computed property value.
Dynamic environment binding opcodes lost the old coordinate warmup.
They were split away from the static coordinate opcodes. Hot closures
and eval-sensitive functions then resolved the same binding by name on
every execution, which regressed JS benchmark throughput badly.
Give each dynamic environment opcode a per-executable coordinate cache
slot. The cache keeps the bytecode stream immutable while letting both
interpreters take a direct declarative environment fast path after the
first lookup. Keep the existing eval invalidation behavior and only warm
caches for declarative-only chains so with environments continue to
observe object shadowing.
Reject cached bytecode that uses the no-cache sentinel for dynamic
environment coordinate cache operands, since execution indexes those
cache arrays unconditionally.
Rebaseline bytecode expectations for the instruction size changes. Add
coverage for with-object shadowing across repeated dynamic lookups and
for rejecting corrupt dynamic environment cache indices.
Store bytecode property lookup caches as tiered handles instead of
eagerly allocating four entries for every cache slot. Each slot starts
empty, grows to one monomorphic entry after the first cacheable lookup,
and promotes to the existing four-entry table when another cache key
appears.
Global variable caches keep one inline property entry because they
usually warm up as global object property accesses. This keeps common
global access allocation-free while still avoiding the inherited
polymorphic cache cost.
Generated asm offsets now point at the lazy property-cache storage and
the inline global entry, and asm fast paths bail out for empty property
cache slots.
Teach Bytecode::Executable to store its instruction stream as either
an owned Vector or a retained Core::ImmutableBytes range. Cached
bytecode materialization now clones the immutable blob owner and lets
the executable point directly into the file-backed cache blob instead
of copying instruction bytes back onto the heap.
Keep a cached instruction data pointer inside the stream wrapper so the
asm interpreter still has a direct hot-path load. Align executable
bytecode payloads in the cache format so mmap-backed instruction
streams satisfy validator and interpreter alignment requirements.
Add separate bytecode instructions for environment lookups that must
stay dynamic, such as eval- and with-sensitive scopes. Keep the
coordinate variants for eagerly resolved declarative environments so
their operands can be treated as immutable at runtime.
This removes cached-coordinate mutation from the interpreter paths and
updates the bytecode expectations for the new dynamic lookup opcodes.
Store compact cache indexes in bytecode instructions instead of raw
pointers to the executable cache vectors. This keeps the instruction
stream independent from heap addresses and removes pointer fixups when
materializing cached bytecode.
Resolve the mutable cache pointers at execution time from the current
Executable. Bytecode test expectations are updated for the smaller cache
operands and resulting instruction offsets.
Store the environment serial number and catch-environment flag in
DeclarativeEnvironment::RareData instead of on every environment.
The serial is only used by global variable caches, and the catch flag
is only needed for catch clause environments, so shaped function
environments should not retain dedicated fields for either value.
Teach the asm global cache fast path to load the serial through rare
data and treat a missing sidecar as serial 0.
Move names, local flags, deletion state, binding lookup, dispose
state, and shape construction bookkeeping out of DeclarativeEnvironment
and into a lazily allocated sidecar. Shaped function environments keep
the shape pointer and value vector directly on the object, but do not
retain empty metadata containers after their shared shape has been
installed.
Keep the asm interpreter fast paths direct by loading local binding
flags from the sidecar only for unshaped environments or dynamic
bindings that come after a shared shape.
Let shaped declarative environments read static binding flags from their
EnvironmentShape instead of retaining a per-environment copy. The local
flag vector now mirrors the local name vector and only stores dynamic
bindings created after the shared shape.
Split EnvironmentShape binding descriptors into separate name and flag
vectors so the shared metadata is compact as well.
Store uninitialized declarative bindings as js_special_empty_value()
in the existing value slot instead of tracking initialization in a
separate bitmap. This removes one bitmap allocation from every
declarative environment while preserving the same TDZ checks.
Update the asm interpreter fast paths to test the value slot directly
and let initialization store the first non-empty value.
Store declarative environment values, names, flags, and initialization
state in separate containers instead of one Binding vector. This packs
per-binding flags into a byte and moves initialization state into an
AK::Bitmap. This reduces the per-environment storage we still need
before binding metadata can move into a shared EnvironmentShape.
Update the asm interpreter offsets and fast binding paths to load
values, flags, and initialization bits from the new storage. Keep
direct binding indices stable across deletion by clearing the slot
while removing the name from the lookup map. Cover that eval deletion
path in the local eval deoptimization tests.
Assert that inline Call frame-top arithmetic does not wrap after adding
the current interpreter stack top, and that raw native frames copy valid
caller lexical and variable environments.
Also check that the raw native ThrowCompletionOr variant index stays in
the expected 0-or-1 range before the asm fast path branches on it.
Assert that asm_helper_to_boolean returns only 0 or 1 before the asm
interpreter branches on the raw helper result. This documents the helper
ABI used by jump and logical-not fast paths without adding code to
release or distribution builds.
Add assertion-only checks for object shapes, binding storage, UTF-16
string backing storage, and iterator receiver shapes before the asm
interpreter relies on those pointers for follow-up loads.
These checks document invariants that are already required by the fast
paths, while still compiling away from release and distribution builds.
Replace assertion-only tag extraction around String helper return
values with assert_tag so release and distribution builds do not carry
code that only exists to feed assertions.
Asserting and non-asserting asmint generation both handle the updated
handlers.
Add debug-only assertions for typed-array cached data and property
iterator cache pointers before the asm fast paths dereference them. The
checks only consume values already loaded for normal execution, so
release and distribution builds still compile them out completely.
asmintgen successfully lowers this file for x86_64 with assertions
enabled.
Add debug-only assertions for value and frame contracts that cross the
asm/C++ boundary. Check resumed inline frames have executable bytecode,
lexical environments exist before boxing, and string builtin helpers
return string values for the fast paths that store their results.
asmintgen successfully lowers this file for x86_64 with assertions
enabled.
Add debug-only checks for relationships between fast-path metadata and
addressing bounds. These catch inconsistent inline-call slot counts,
argument padding, and flattened iterator cache lengths before the asm
path uses them for addressing.
asmintgen successfully lowers this file for x86_64 with assertions
enabled.
Add debug-only assertions around inline Call frame construction and
state restoration. The checks cover executable metadata, stack cursors,
realm/environment pointers, constant storage, raw native entry points,
and the VM state reloaded after helper calls.
asmintgen successfully lowers this file for x86_64 with assertions
enabled.
Add debug-only asmint assertions for cache and backing-store pointers
that fast paths dereference after metadata has selected a cache hit.
These checks catch stale property/global caches, broken environment
coordinates, and inconsistent indexed storage state without changing
release or distribution builds.
asmintgen successfully lowers this file for x86_64 with assertions
enabled.
Track whether a function needs environment-backed this resolution
separately from whether it needs to allocate its own function
environment. Arrow functions that only capture lexical this can now
resolve through the outer environment without allocating an empty
function environment for every call.
Keep the asm Call path conservative by routing functions that still need
lexical-this resolution through the C++ inline-call helper, so the call
receiver is not cached as the arrow function's this value.
Microbenchmark:
function makeLexicalThisArrow() {
return () => this.value;
}
let object = { value: 1, makeLexicalThisArrow };
let fn = object.makeLexicalThisArrow();
for (let i = 0; i < 20_000_000; ++i)
fn();
Measured with the same Release build toggling this patch:
baseline: 1069.2 ms mean over 12 runs
optimized: 501.2 ms mean over 12 runs
speedup: 2.13 times faster
asmint.asm no longer references any positional temp register name --
every handler and macro declares its temporaries by name with `temp` /
`ftemp` and lets the register allocator place them. Migrate the last
two macros holding out:
* dispatch_current uses a macro-local `opcode` temp for the load8 +
indirect jmp.
* pop_inline_frame_and_resume names its return-pc, dst-index, value-
address, vm-pointer, and executable temps explicitly.
With nothing left referring to the positional aliases, drop the
tN / ftN -> physical-register fallback from registers::resolve_register
and update the DSL reference comments at the top of asmint.asm and in
main.rs to describe the named-temp model. The two pre-existing codegen
tests that probed the old positional behavior get rewritten to use the
post-allocation physical-register names directly, since that is now
the actual contract of resolve_op.
The Call handler is the asm interpreter's hot path for inline JS-to-JS
dispatch, and it carries roughly fifty distinct named values across
its body (callee object, packed metadata word, formal/passed/total
counts, two stack-side pointers, the argument-copy cursor, the
script-or-module pair-load scratch, the native return value and
variant tag, ...).
The native-exception path adds a small `mov helper_arg, native_return`
bridge between call_raw_native (output pinned to rax by
fixed_operands) and call_helper (input pinned to rcx by
fixed_operands), since the two ABIs land their values in different
registers and the explicit mov is the cleanest way to express the
hand-off.
Convert the for-in iteration fast path to named temps. The handler
threads ~25 distinct values through its body (the iterator value,
its tag, the unboxed iterator, the fast-path discriminator, the
property cache pointer, the cached and current shapes, the receiver,
the dictionary-generation pair, the indexed-count and next-indexed
counters, the named-index, the named-key data pointer, and a few
scratch slots), so the migrated form is much easier to read than the
positional spaghetti where t0..t8 each meant five different things in
different basic blocks.
Convert the array / typed-array load fast paths in GetByValue, the
mirror of PutByValue. Roughly the same shape: ~18 named GPR temps
and a slot_dbl FPR temp covering the loaded value, the bytes-or-int
raw integer view, the address scratch, and the negative-zero compare.
The .ta_float32 / .ta_float64 paths become readable: where the
positional code had "Exclude negative zero early (t1 gets clobbered
by double_to_int32)" because t1 had to do double duty, the migrated
form names slot, slot_dbl, neg_zero, and raw separately and the
allocator lays them out without the implicit clobber dance.
Convert the array / typed-array store fast paths in PutByValue --
the largest single handler so far at 150+ positional t-uses -- to
declare its 19 GPR temps and 1 FPR temp by name (kind, base, prop,
index, obj, flags, storage_kind, size, elements, src, capacity_addr,
capacity, slot, empty_tag, kind_byte, addr, src_int32, max, result,
src_dbl, plus base_tag/prop_tag scratches reused across paths).
The migrated body removes the "save in t0 before load_operand
clobbers it" / "compute store address before check_is_double clobbers
t4" maneuvers that the positional code needed -- the allocator
handles those constraints automatically.
Convert the load_primitive_string_utf16_code_unit macro to take its
inputs (string pointer, index) and output (code_unit) as explicit
parameters, plus migrate CallBuiltinStringFromCharCode,
CallBuiltinStringPrototypeCharCodeAt, and
CallBuiltinStringPrototypeCharAt.
The macro previously hard-coded inputs to t2/t4 and output to t0,
which forced callers to remember those slot assignments and to wrap
the call_helper interactions around them. The migrated form -- where
the caller names what it wants in each slot -- removes another
"clobbers t3, t5" comment.
Convert the global variable IC and the array-length fast path to
named temps. These three handlers are large -- GetGlobal and
SetGlobal carry ~20 named values across each path (realm pointer,
global object, declarative environment, cache pointer, two serial
numbers, the shape, the cached shape, the dictionary generations,
the property offset, the named-properties pointer, the loaded value
and tag, plus environment-binding state) -- and the migration makes
those values easy to read instead of having to remember which slot
each piece of state was occupying.
GetLength's "magical" int-to-double widening picks up a named
temp for the sign-bit check and the result, which removes another
"NB: load_operand clobbers t0" comment.
Convert the inline-cache fast paths for property load/store to named
temps. Both handlers carry a substantial amount of state across the
ic-hit check (the boxed base value, its tag, the unboxed Object*,
the shape, the PLC pointer, the cached shape and prototype, the
property offset, the current and cached dictionary generations, the
named-properties pointer, and the loaded value), so giving each
piece an explicit name removes a real readability burden. The IC-miss
path uses the new 3-operand call_interp form.
Notably, PutById's old code carried a "save property offset in t4
before load_operand clobbers t0 (rax)" maneuver -- that is exactly
the kind of cross-cut the allocator handles automatically once the
offset is named. The migrated handler just keeps using `prop_offset`
across the load_operand and the store, no scratch shuffle needed.
