This doesn't affect interpreter size directly, but let's inform the
compiler that we're not terribly worried about code using the `with`
statement in JS.
While we're in the bytecode compiler, we want to know which type of
Operand we're dealing with, but once we've generated the bytecode
stream, we only ever need its index.
This patch simplifies Operand by removing the aarch64 bitfield hacks
and makes it 32-bit on all platforms. We keep 3 type bits in the high
bits of the index while compiling, and then zero them out when
flattening the final bytecode stream.
This makes bytecode more compact on x86_64, and avoids bit twiddling
on aarch64. Everyone wins something!
When stringifying bytecode for debugging output, we now have an API in
Executable that can look at a raw operand index and tell you what type
of operand it was, based on known quantities of each type in the stack
frame.
In our process architecture, there's only ever one JS::VM per process.
This allows us to have a VM::the() singleton getter that optimizes
down to a single global access everywhere.
Seeing 1-2% speed-up on all JS benchmarks from this.
This allows us to use the bytecode implementation of await, which
correctly suspends execution contexts and handles completion
injections.
This gains us 4 test262 tests around mutating Array.fromAsync's
iterable whilst it's suspended as well.
This is also one step towards removing spin_until, which the
non-bytecode implementation of await uses.
```
Duration:
-5.98s
Summary:
Diff Tests:
+4 ✅ -4 ❌
Diff Tests:
[...]/Array/fromAsync/asyncitems-array-add-to-singleton.js ❌ -> ✅
[...]/Array/fromAsync/asyncitems-array-add.js ❌ -> ✅
[...]/Array/fromAsync/asyncitems-array-mutate.js ❌ -> ✅
[...]/Array/fromAsync/asyncitems-array-remove.js ❌ -> ✅
```
This hosts the ability to compile and run JavaScript to implement
native functions. This is particularly useful for any native function
that is not a normal function, for example async functions such as
Array.fromAsync, which require yielding.
These functions are not allowed to observe anything from outside their
environment. Any global identifiers will instead be assumed to be a
reference to an abstract operation or a constant. The generator will
inject the appropriate bytecode if the name of the global identifier
matches a known name. Anything else will cause a code generation error.
All the data we need for compilation is in SharedFunctionInstanceData,
so we shouldn't depend on ECMAScriptFunctionObject.
Allows NativeJavaScriptBackedFunction to compile bytecode.
This commit adds a new Bytecode.def file that describes all the LibJS
bytecode instructions.
From this, we are able to generate the full declarations for all C++
bytecode instruction classes, as well as their serialization code.
Note that some of the bytecode compiler was updated since instructions
no longer have default constructor arguments.
The big immediate benefit here is that we lose a couple thousand lines
of hand-written C++ code. Going forward, this also allows us to do more
tooling for the bytecode VM, now that we have an authoritative
description of its instructions.
Key things to know about:
- Instructions can inherit from one another. At the moment, everything
simply inherits from the base "Instruction".
- @terminator means the instruction terminates a basic block.
- @nothrow means the instruction cannot throw. This affects how the
interpreter interacts with it.
- Variable-length instructions are automatically supported. Just put an
array of something as the last field of the instruction.
- The m_length field is magical. If present, it will be populated with
the full length of the instruction. This is used for variable-length
instructions.
With this change, `GetIterator` no longer GC-allocates an
`IteratorRecord`. Instead, it stores the iterator record fields in
bytecode registers. This avoids per-iteration allocations in patterns
like: `for (let [x] of array) {}`.
`IteratorRecord` now inherits from `IteratorRecordImpl`, which holds the
iteration state. This allows the existing iteration helpers
(`iterator_next()`, `iterator_step()`, etc.) operate on both the
GC-allocated and the register-backed forms.
Microbenchmarks:
1.1x array-destructuring-assignment-rest.js
1.226x array-destructuring-assignment.js
This reverts commit cdcbbcf48b.
It made MicroBench/call-*-args.js faster, but some of the macro
benchmarks got significantly slower on macOS, so let's revert until we
understand it better.
Instead of always checking if we're about to return an empty completion
value in Interpreter::run_executable(), we now coerce empty completions
to the undefined value earlier instead.
This simplifies the most common path through run_executable(), giving us
a small speedup.
Instead of using this span, we can just use the getter that calculates
the base of the register/constant/local/argument array based on the
ExecutionContext's own address.
We don't need to return two values; running an executable only ever
produces a throw completion, or a normal completion, i.e a Value.
This necessitated a few minor changes, such as adding a way to check
if a JS::Cell is a GeneratorResult.
By handling call instructions in an inline (C++) function, we were
breaking the alloca() optimization and adding stack overhead. We fix
this by using a macro instead. It looks awful but it works.
1.07x speedup on MicroBench/call-00-args.js
This simplifies function entry/exit and lets us just walk away from the
used ExecutionContext instead of resetting a bunch of its state when
returning control to the caller.
This commits puts the strict mode flag in the header of every bytecode
instruction. This allows us to check for strict mode without looking at
the currently running execution context.
The GC::Ref smart pointer is always non-null, so there's no need for it
to be convertible to bool.
This exposed a small number of unnecessary null checks which we remove.
When an object becomes too big (currently 64 properties or more), we
change its shape to a dictionary and don't do any further transitions.
However, this means the Shape of the object no longer changes, so the
cache invalidation check of `current_shape != cache.shape` is no longer
a valid check.
This fixes that by keeping track of a generation number for the Shape
both on the Shape object and in the cache, allowing that to be checked
instead of the Shape identity. The generation is incremented whenever
the dictionary is mutated.
Fixes stale cache lookups on Gmail preventing emails from being
displayed.
I was not able to produce a reproduction for this, plus the generation
count was over the 20k mark on Gmail.
We also make the code a bit more generic by making callers provide
(templated) callbacks that produce the property name and base expression
string if any.
This makes the instanceof operator signficantly faster by avoiding a
generic function call to @@hasInstance unless it has been overridden.
1.15x speed-up on Octane/earley-boyer.js
We already had fast paths for Add, Sub and Mul. Might as well do Div.
1.18x speed-up on this micro-benchmark:
(() => {
let a = 1234;
for (let i = 0; i < 100_000_000; ++i)
a / a;
})()
This is only used to specify how a property is being added to an object
by Put* instructions, so let's call it PutKind.
Also add an enumeration X macro for it to prepare for upcoming
specializations.
This gets rid of a lot of pointer chasing from interpreter to executable
to identifier table to the actual identifier.
1.05x speed-up on Kraken/ai-astar.js
These will generally be cached the vast majority of the time except on
first encounter, and sprinkling [[likely]] gives us a nice boost.
1.10x speed-up on this micro-benchmark:
(() => {
var a = 3;
for (let i = 0; i < 100_000_000; ++i) { a; }
eval("");
})();
Instead of converting them to doubles and doing double math, just do the
arithmetic operation in i64 space instead.
This gives us a ~1.25x speed-up on this kind of micro-benchmark:
(() => {
let a = -2124299999;
for (let i = 0; i < 100_000_000; ++i) {
a + a;
}
})()
Same idea for Add, Sub, and Mul.
There's a fair bit of overflowing Int32 arithmetic in some of the
JetStream benchmarks, and this seems like an obvious improvement.
This change implements a part responsible for this invariant in a more
efficient way:
"Enumerating the properties of the target object includes enumerating
properties of its prototype, and the prototype of the prototype, and so
on, recursively; but a property of a prototype is not processed if it
has the same name as a property that has already been processed by the
iterator's next method."
Previously we inserted `(key, enumerable)` pairs into an
`OrderedHashTable`. That always built and maintained a hash table, even
when no prototype-level filtering was needed.
Now we:
- Collect only enumerable keys into `Vector<PropertyKey>`.
- Track `seen_non_enumerable_properties` so a non-enumerable own
property still shadows prototype properties with the same name.
- Lazily materialize `HashTable<PropertyKey>` only if we encounter an
enumerable property on a prototype and must check for duplicates. In
the common case materialization is avoided, because default Object or
Array prototype properties are non-enumerable.
Before this change, PropertyNameIterator (used by for..in) and
`Object::enumerable_own_property_names()` (used by `Object.keys()`,
`Object.values()`, and `Object.entries()`) enumerated an object's own
enumerable properties exactly as the spec prescribes:
- Call `internal_own_property_keys()`, allocating a list of JS::Value
keys.
- For each key, call internal_get_own_property() to obtain a
descriptor and check `[[Enumerable]]`.
While that is required in the general case (e.g. for Proxy objects or
platform/exotic objects that override `[[OwnPropertyKeys]]`), it's
overkill for ordinary JS objects that store their own properties in the
shape table and indexed-properties storage.
This change introduces `for_each_own_property_with_enumerability()`,
which, for objects where
`eligible_for_own_property_enumeration_fast_path()` is `true`, lets us
read the enumerability directly from shape metadata (and from
indexed-properties storage) without a per-property descriptor lookup.
When we cannot avoid `internal_get_own_property()`, we still
benefit by skipping the temporary `Vector<Value>` of keys and avoiding
the unnecessary round-trip between PropertyKey and Value.
- Capture PrototypeChainValidity before invoking `internal_get()`. A
getter may mutate the prototype chain (e.g., delete itself). Capturing
earlier ensures such mutations invalidate the cached entry and prevent
stale GetById hits.
- When caching, take PrototypeChainValidity from the base object
(receiver), not from the prototype where the property was found.
Otherwise, changes to an intermediate prototype between the base
object and the cached prototype object go unnoticed, leading to
incorrect cache hits.
We already had IC support in PutById for the following cases:
- Changing an existing own property
- Calling a setter located in the prototype chain
This was enough to speed up code where structurally identical objects
(same shape) are processed in a loop:
```js
const arr = [{ a: 1 }, { a: 2 }, { a: 3 }];
for (let obj of arr) {
obj.a += 1;
}
```
However, creating structurally identical objects in a loop was still
slow:
```js
for (let i = 0; i < 10_000_000; i++) {
const o = {};
o.a = 1;
o.b = 2;
o.c = 3;
}
```
This change addresses that by adding a new IC type that caches both the
source and target shapes, allowing property additions to be fast-pathed
by directly jumping to the shape that already includes the new property.
Previously, PutById constructed a PropertyKey from the identifier,
which coerced numeric-like strings to numbers. This moves that decision
to bytecode generation: the bytecode generator now emits PutByNumericId
for numeric keys and PutById for string keys. This removes per-execution
parsing from the interpreter.
1.4x speedup on the following microbenchmark:
```js
const o = {};
for (let i = 0; i < 10_000_000; i++) {
o.a = 1;
o.b = 2;
o.c = 3;
}
```
This has quite a lot of fall out. But the majority of it is just type or
UDL substitution, where the changes just fall through to other function
calls.
By changing property key storage to UTF-16, the main affected areas are:
* NativeFunction names must now be UTF-16
* Bytecode identifiers must now be UTF-16
* Module/binding names must now be UTF-16
Before this change each built-in iterator object has a boolean
`m_next_method_was_redefined`. If user code later changed the iterator’s
prototype (e.g. `Object.setPrototypeOf()`), we still believed the
built-in fast-path was safe and skipped the user supplied override,
producing wrong results.
With this change
`BuiltinIterator::as_builtin_iterator_if_next_is_not_redefined()` looks
up the current `next` property and verifies that it is still the
built-in native function.
This commit adds the minimal export macros needed to run js.exe on
windows. A followup commit is planned to move to explicit export
entirely.
A static_assert for the size of a struct is also ifdef'ed out as the
semantics around object layout and inheritance are different on MSVC abi
and the struct IteratorRecord ends up being 40 bytes not 32.
Fixes a bug that reproduces with the following steps:
1. Create an object with a getter for property "a" in its prototype,
where the getter adds an "a" property to the object itself.
2. Call the "a" getter in a loop for the first time. This triggers
caching of metadata indicating that the "a" property is located in
the prototype chain.
3. Call the "a" getter in a loop for the second time. Oops, the cache
says the getter is in the prototype chain, but the object now
also has its own "a" property that was added by the first getter
call.
- Avoids unnecessary conversions between StringOrSymbol and PropertyKey
on the hot path of property access.
- Simplifies the code by removing StringOrSymbol and using PropertyKey
directly. There was no reason to have a separate StringOrSymbol type
representing the same data as PropertyKey, just with the index key
stored as a string.
PropertyKey has been updated to use a tagged pointer instead of a
Variant, so it still occupies 8 bytes, same as StringOrSymbol.
12% improvement on JetStream/gcc-loops.cpp.js
12% improvement on MicroBench/object-assign.js
7% improvement on MicroBench/object-keys.js
By doing that we avoid lots of `PropertyKey` -> `Value` -> `PropertyKey`
transforms, which are quite expensive because of underlying
`FlyString` -> `PrimitiveString` -> `FlyString` conversions.
10% improvement on MicroBench/object-keys.js
This commit adds a fast path for putting values into a TypedArray of an
integer type, when the value being put in is a double. This leads to a
6% speedup on JetStream/gcc-loops.js.
81b6a11 regressed correctness by always bypassing the `next()` method
resolution for built-in iterators, causing incorrect behavior when
`next()` was redefined on built-in prototypes. This change fixes the
issue by storing a flag on built-in prototypes indicating whether
`next()` has ever been redefined.
https://tc39.es/ecma262/#sec-jobs specifies that we should only be
running queued promise jobs and host-defined cleanup when the
execution context stack is empty. It is asserted to _not_ be empty
the line above, so remove it.
No impact on test262 or our test suites, Interpreter::run_executable
is already (incorrectly) performing this unconditionally.
run_promise_jobs also happens to do nothing when LibJS is embedded
into LibWeb.
Instead of monomorphic (1 shape), GetById inline caches are now
polymorphic (4 shapes).
This improves inline cache hit rates greatly on most web JavaScript.
For example, Speedometer 2.1 sees 88% -> 97% cache hit rate improvement.
1.71x speedup on MicroBench/pic-get-own.js
1.82x speedup on MicroBench/pic-get-pchain.js
This is *extremely* common on the web, but barely shows up at all in
JavaScript benchmarks.
A typical example is setting Element.innerHTML on a HTMLDivElement.
HTMLDivElement doesn't have innerHTML, so it has to travel up the
prototype chain until it finds it.
Before this change, we didn't cache this at all, so we had to travel
the prototype chain every time a setter like this was used.
We now use the same mechanism we already had for GetBydId and cache
PutById setter accesses in the prototype chain as well.
1.74x speedup on MicroBench/setter-in-prototype-chain.js
`var` bindings are never in the temporal dead zone (TDZ), and so we
know accessing them will not throw.
We now take advantage of this by having a specialized environment
binding value getter that doesn't check for exceptional cases.
1.08x speedup on JetStream.
Before this change, setting a global would end up as SetLexicalBinding.
That instruction always failed to cache the access if the global was a
property of the global object.
1.14x speedup on Octane/earley-boyer.js
2.04x speedup on MicroBench/for-of.js
Note that MicroBench/for-of.js was more of a "set global" benchmark
before this. After this change, it's actually a for..of benchmark. :^)
We were spending a lot of time removing each property name from the
iterator's underlying HashMap while iterating over it. This wasn't
actually necessary, so let's stop doing it and instead just iterate
over the property names with a stored HashTable iterator.
1.10x speedup on MicroBench/for-in-indexed-properties.js
Before this change, we would call [[OwnPropertyKeys]] on the target
objects, then convert the returned keys from Value into PropertyKey.
Then, when actually iterating, we'd convert them back into Value again.
This was particularly costly for numeric property keys, since we had
to go through string-from-number construction.
Now, we simply keep the original values returned by [[OwnPropertyKeys]]
around and use them for the enumeration.
1.09x speedup on MicroBench/for-in-indexed-properties.js
1.01x speedup on MicroBench/for-in-named-properties.js
I was investigating an optimization in this area, and while it
didn't seem to have a noticable improvement, it still seems
useful to apply this change.
Even though this code was already optimized to re-use a single result
object, returning { value, done } directly in output parameters still
provides a substantial speedup.
1.21x speedup on MicroBench/for-in-indexed-properties.js
Apply a little ensure_capacity() to avoid excessive rehashing of the
property key table when enumerating a large number of properties.
1.23x speedup on MicroBench/for-in-indexed-properties.js
...by avoiding `{ value, done }` iterator result value allocation. This
change applies the same otimization 81b6a11 added for `for..in` and
`for..of`.
Makes following micro benchmark go 22% faster on my computer:
```js
function f() {
const arr = [];
for (let i = 0; i < 10_000_000; i++) {
arr.push([i]);
}
let sum = 0;
for (let [i] of arr) {
sum += i;
}
}
f();
```
Introduce special instruction for `for..of` and `for..in` loop that
skips `{ value, done }` result object allocation if iterator is builtin
(array, map, set, string). This reduces GC pressure significantly and
avoids extracting the `value` and `done` properties.
This change makes this micro benchmark 48% faster on my computer:
```js
const arr = new Array(10_000_000);
let counter = 0;
for (let _ of arr) {
counter++;
}
```
This reverts commit 36bb2824a6.
Although this was faster on my M3 MacBook Pro, other Apple machines
disagree, including our benchmark runner. So let's revert it.
This is a simple trick to generate better native code for access to
registers, locals, and constants. Before this change, each access had
to first dereference the member pointer in Interpreter, and then get to
the values. Now we always have a pointer directly to the values on hand.
Here's how it looks:
class StackFrame {
public:
Value get(Operand) const;
void set(Operand, Value);
private:
Value m_values[];
};
And we just place one of these as a window on top of the execution
context's array of values (registers, locals, and constants).
Getting the running_execution_context() already verifies that the
execution context stack is non-empty, we don't need to do it separately
here as well.
The old accumulator register is really only used to pass the end
completion to the caller of run_bytecode() nowadays. As such, we don't
need to cache a pointer to it for fast access. One less thing to do
on run_bytecode() entry.
This way it's always automatically correct, and we don't have to
manually flush it in push_execution_context().
~7% speedup on the MicroBench/call* tests :^)
Instead of letting every [[Call]] implementation allocate an
ExecutionContext, we now make that a responsibility of the caller.
The main point of this exercise is to allow the Call instruction
to write function arguments directly into the callee ExecutionContext
instead of copying them later.
This makes function calls significantly faster:
- 10-20% faster on micro-benchmarks (depending on argument count)
- 4% speedup on Kraken
- 2% speedup on Octane
- 5% speedup on JetStream
By doing that we avoid doing separate allocation for each such vector,
which was really expensive on js heavy websites. For example this change
helps to get EC allocation down from ~17% to ~2% on Google Maps. This
comes at cost of adding extra complexity to custom execution context
allocator, because EC no longer has fixed size and we need to maintain
a list of buckets.
This is better because:
- Better data locality
- Allocate vector for registers+constants+locals+arguments in one go
instead of allocating two vectors separately
This allows us to remove the BoundFunction::m_name field, which we
were initializing with a formatted FlyString on every function binding,
despite never using it for anything.
Before this change, we were inlining this function after every
handler for instructions that could throw.
Forcing it out-of-line shrinks the main bytecode interpreter by 15%
and yields a decent 2.5% speedup on JetStream/gcc-loops.cpp.js
We can use the index's invalid state to signal an empty optional.
This makes Optional<StringTableIndex> 4 bytes instead of 8,
shrinking every bytecode instruction that uses these.
The special empty value (that we use for array holes, Optional<Value>
when empty and a few other other placeholder/sentinel tasks) still
exists, but you now create one via JS::js_special_empty_value() and
check for it with Value::is_special_empty_value().
The main idea here is to make it very unlikely to accidentally create an
unexpected special empty value.
Instead of returning internal generator results as ordinary JS::Objects
with properties, we now use GeneratorResult and CompletionCell which
both inherit from Cell directly and allow efficient access to state.
1.59x speedup on JetStream3/lazy-collections.js :^)
This works because at the end of the finally chunk, a
ContinuePendingUnwind is generated which copies the saved return value
register into the return value register. In cases where
ContinuePendingUnwind is not generated such as when there is a break
statement in the finally block, the fonction will return undefined which
is consistent with V8 and SpiderMonkey.
Before this change, we would enumerate all the keys with
[[OwnPropertyKeys]], and then do [[GetOwnPropertyDescriptor]] twice for
each key as we went through them.
We now only do one [[GetOwnPropertyDescriptor]] per key, which
drastically reduces the number of proxy traps when those are involved.
The new trap sequence matches what you get with V8, so I don't think
anyone will be unpleasantly surprised here.
Instead of creating a new iterator result Object for every step of
for..in iteration, we can create a single object up front and reuse it
for every step. This avoids generating a bunch of garbage that isn't
observable by author code anyway.
We can also reuse the existing premade shape for these objects.
Instead of pruning as-we-go, which means a ton of hash lookups,
we now only do a single pass to prune all non-enumerable keys when
setting up for for..in iteration.
This is no longer done. One of the comments is also innacurate for a
second reason - the call stack is never empty in that case, and is
verified as such only a few lines above.
From what I understand, the suspension steps are not required now,
or in the future for our implementation, or any other. The intent
is already implemented in the spec pushing on another execution
context to the stack and leaving the running execution context as-is.
The resume steps are a slightly different story as there is some subtle
behavior which the spec is trying to convey where some custom logic may
need to be done when one execution context changes from one to another.
It may be worth implementing those steps at a later point in time so
that this behavior is a bit easier to follow in those cases.
To make the situation more confusing - from what I can gather from the
spec, not all cases that the spec mentions resume actually means
anything normative. Resume is only _actually_ needed in a limited set
of locations.
For now, let's just remove the unneeded FIXMEs that indicate that there
is something to be done for the suspension steps, as there is not, and
leave the resume steps as is.
Resulting in a massive rename across almost everywhere! Alongside the
namespace change, we now have the following names:
* JS::NonnullGCPtr -> GC::Ref
* JS::GCPtr -> GC::Ptr
* JS::HeapFunction -> GC::Function
* JS::CellImpl -> GC::Cell
* JS::Handle -> GC::Root
The main motivation behind this is to remove JS specifics of the Realm
from the implementation of the Heap.
As a side effect of this change, this is a bit nicer to read than the
previous approach, and in my opinion, also makes it a little more clear
that this method is specific to a JavaScript Realm.
When the cached value was not an accessor, it was simply ignored.
This is the value we really want, so we can just return it.
Shows up to 5x improvements on some benchmarks,
and 1.4x in general js-benchmarks.
