And use it to _correctly_ implement state saving for generators.
Prior to this, we were capturing the caller frame, which is completely
irrelevant to the generator frame.
To support situations like this:
function foo() { throw 1; }
try {
foo();
} catch (e) {
}
Each unwind context now keeps track of its origin executable.
When an exception is thrown, we return from run() immediately if the
nearest unwind context isn't in the current executable.
This causes a natural unwind to the point where we find the
catch/finally block(s) to jump into.
We were missing some "break" statements, causing us to actually finish
executing everything within "try" blocks before actually jumping to the
"catch" and/or "finally" blocks.
This is a specialized string table for storing identifiers only.
Identifiers are always FlyStrings, which makes many common operations
faster by allowing O(1) comparison.
This gives FunctionNode a "might need arguments object" boolean flag and
sets it based on the simplest possible heuristic for this: if we
encounter an identifier called "arguments" or "eval" up to the next
(nested) function declaration or expression, we won't need an arguments
object. Otherwise, we *might* need one - the final decision is made in
the FunctionDeclarationInstantiation AO.
Now, this is obviously not perfect. Even if you avoid eval, something
like `foo.arguments` will still trigger a false positive - but it's a
start and already massively cuts down on needlessly allocated objects,
especially in real-world code that is often minified, and so a full
"arguments" identifier will be an actual arguments object more often
than not.
To illustrate the actual impact of this change, here's the number of
allocated arguments objects during a full test-js run:
Before:
- Unmapped arguments objects: 78765
- Mapped arguments objects: 2455
After:
- Unmapped arguments objects: 18
- Mapped arguments objects: 37
This results in a ~5% speedup of test-js on my Linux host machine, and
about 3.5% on i686 Serenity in QEMU (warm runs, average of 5).
The following microbenchmark (calling an empty function 1M times) runs
25% faster on Linux and 45% on Serenity:
function foo() {}
for (var i = 0; i < 1_000_000; ++i)
foo();
test262 reports no changes in either direction, apart from a speedup :^)
Before this we used an ad-hoc combination of references and 'variables'
stored in a hashmap. This worked in most cases but is not spec like.
Additionally hoisting, dynamically naming functions and scope analysis
was not done properly.
This patch fixes all of that by:
- Implement BindingInitialization for destructuring assignment.
- Implementing a new ScopePusher which tracks the lexical and var
scoped declarations. This hoists functions to the top level if no
lexical declaration name overlaps. Furthermore we do checking of
redeclarations in the ScopePusher now requiring less checks all over
the place.
- Add methods for parsing the directives and statement lists instead
of having that code duplicated in multiple places. This allows
declarations to pushed to the appropriate scope more easily.
- Remove the non spec way of storing 'variables' in
DeclarativeEnvironment and make Reference follow the spec instead of
checking both the bindings and 'variables'.
- Remove all scoping related things from the Interpreter. And instead
use environments as specified by the spec. This also includes fixing
that NativeFunctions did not produce a valid FunctionEnvironment
which could cause issues with callbacks and eval. All
FunctionObjects now have a valid NewFunctionEnvironment
implementation.
- Remove execute_statements from Interpreter and instead use
ASTNode::execute everywhere this simplifies AST.cpp as you no longer
need to worry about which method to call.
- Make ScopeNodes setup their own environment. This uses four
different methods specified by the spec
{Block, Function, Eval, Global}DeclarationInstantiation with the
annexB extensions.
- Implement and use NamedEvaluation where specified.
Additionally there are fixes to things exposed by these changes to eval,
{for, for-in, for-of} loops and assignment.
Finally it also fixes some tests in test-js which where passing before
but not now that we have correct behavior :^).
The old name is the result of the perhaps somewhat confusingly named
abstract operation OrdinaryFunctionCreate(), which creates an "ordinary
object" (https://tc39.es/ecma262/#ordinary-object) in contrast to an
"exotic object" (https://tc39.es/ecma262/#exotic-object).
However, the term "Ordinary Function" is not used anywhere in the spec,
instead the created object is referred to as an "ECMAScript Function
Object" (https://tc39.es/ecma262/#sec-ecmascript-function-objects), so
let's call it that.
The "ordinary" vs. "exotic" distinction is important because there are
also "Built-in Function Objects", which can be either implemented as
ordinary ECMAScript function objects, or as exotic objects (our
NativeFunction).
More work needs to be done to move a lot of infrastructure to
ECMAScriptFunctionObject in order to make FunctionObject nothing more
than an interface for objects that implement [[Call]] and optionally
[[Construct]].
This is where the spec wants to have it. Requires a couple of hacks as
currently everything that needs a Realm actually has a GlobalObject, so
we need to go via the Interpreter.
The test262 tests under RegExp/property-escapes/generated will invoke
Reflect.apply with up to 10,000 arguments at a time. In LibJS, when the
call stack reached VM::call_internal, we transfer those arguments from
a MarkedValueList to the execution context's arguments Vector.
Because these types differ (MarkedValueList is a Vector<Value, 32>), the
arguments are copied rather than moved. By changing the arguments vector
to a MarkedValueList, we can properly move the passed arguments over.
This shaves about 2 seconds off the following test262 test (from 15sec):
RegExp/property-escapes/generated/General_Category_-_Decimal_Number.js
RegExpInitialize specifies how the pattern string should be created
before passing it to [[RegExpMatcher]]. Rather than passing it as-is,
the string should be converted to code points and back to a "List" (if
the Unicode flag is present), or as a "List" of UTF-16 code units.
Further. the spec requires that we keep both the original pattern string
and this parsed string in the RegExp object.
The caveat is that the LibRegex parser further requires any multi-byte
code units to be escaped (as "\unnnn"). Otherwise, the code unit is
recognized as individual UTF-8 bytes.
This removes all usages of the non-standard put helper method and
replaces all of it's usages with the specification required alternative
or with define_direct_property where appropriate.
This removes all usages of the non-standard define_property helper
method and replaces all it's usages with the specification required
alternative or with define_direct_property where appropriate.
This is a huge patch, I know. In hindsight this perhaps could've been
done slightly more incremental, but I started and then fixed everything
until it worked, and here we are. I tried splitting of some completely
unrelated changes into separate commits, however. Anyway.
This is a rewrite of most of Object, and by extension large parts of
Array, Proxy, Reflect, String, TypedArray, and some other things.
What we already had worked fine for about 90% of things, but getting the
last 10% right proved to be increasingly difficult with the current code
that sort of grew organically and is only very loosely based on the
spec - this became especially obvious when we started fixing a large
number of test262 failures.
Key changes include:
- 1:1 matching function names and parameters of all object-related
functions, to avoid ambiguity. Previously we had things like put(),
which the spec doesn't have - as a result it wasn't always clear which
need to be used.
- Better separation between object abstract operations and internal
methods - the former are always the same, the latter can be overridden
(and are therefore virtual). The internal methods (i.e. [[Foo]] in the
spec) are now prefixed with 'internal_' for clarity - again, it was
previously not always clear which AO a certain method represents,
get() could've been both Get and [[Get]] (I don't know which one it
was closer to right now).
Note that some of the old names have been kept until all code relying
on them is updated, but they are now simple wrappers around the
closest matching standard abstract operation.
- Simplifications of the storage layer: functions that write values to
storage are now prefixed with 'storage_' to make their purpose clear,
and as they are not part of the spec they should not contain any steps
specified by it. Much functionality is now covered by the layers above
it and was removed (e.g. handling of accessors, attribute checks).
- PropertyAttributes has been greatly simplified, and is being replaced
by PropertyDescriptor - a concept similar to the current
implementation, but more aligned with the actual spec. See the commit
message of the previous commit where it was introduced for details.
- As a bonus, and since I had to look at the spec a whole lot anyway, I
introduced more inline comments with the exact steps from the spec -
this makes it super easy to verify correctness.
- East-const all the things.
As a result of all of this, things are much more correct but a bit
slower now. Retaining speed wasn't a consideration at all, I have done
no profiling of the new code - there might be low hanging fruits, which
we can then harvest separately.
Special thanks to Idan for helping me with this by tracking down bugs,
updating everything outside of LibJS to work with these changes (LibWeb,
Spreadsheet, HackStudio), as well as providing countless patches to fix
regressions I introduced - there still are very few (we got it down to
5), but we also get many new passing test262 tests in return. :^)
Co-authored-by: Idan Horowitz <idan.horowitz@gmail.com>
Specifically, this now explicitly takes the length, adds missing
exceptions checks to calls with user-supplied lengths, takes and uses
the prototype argument, and fixes some spec non-conformance in
ArrayConstructor and its native functions around the use of ArrayCreate
The parser doesn't always track lexical scopes correctly, so let's not
rely on that for direct argument loading.
This reverts the LoadArguments bytecode instruction as well. We can
bring these things back when the parser can reliably tell us that
a given Identifier is indeed a function argument.
These represent the outermost scope in the environment record
hierarchy. The spec says they should be a "composite" of two things:
- An ObjectEnvironmentRecord wrapping the global object
- A DeclarativeEnvironmentRecord for other declarations
It's not yet clear to me how this should work, so this patch only
implements the first part, an object record wrapping the global object.
To better follow the spec, we need to distinguish between the current
execution context's lexical environment and variable environment.
This patch moves us to having two record pointers, although both of
them point at the same environment records for now.
This patch makes the following name changes:
- ScopeObject => EnvironmentRecord
- LexicalEnvironment => DeclarativeEnvironmentRecord
- WithScope => ObjectEnvironmentRecord
This now matches the spec's OrdinaryObjectCreate() across the board:
instead of implicitly setting the created object's prototype to
%Object.prototype% and then in many cases setting it to a nullptr right
away, it now has an 'Object* prototype' parameter with _no default
value_. This makes the code easier to compare with the spec, very clear
in terms of what prototype is being used as well as avoiding unnecessary
shape transitions.
Also fixes a couple of cases were we weren't setting the correct
prototype.
There's no reason to assume that the object would not be empty (as in
having own properties), so let's follow our existing pattern of
Type::create(...) and simply call it 'create'.
This commit adds a bunch of passes, the most interesting of which is a
pass that merges blocks together, and a pass that places blocks that
flow into each other next to each other, and a very simply pass that
removes duplicate basic blocks.
Note that this does not remove the jump at the end of each block in that
pass to avoid scope creep in the passes.
These are pretty hairy if someone forgets to override one, as the
catchall function in Instruction will keep calling itself over and over
again, leading to really hard-to-debug situations.
This commit implements parsing for `yield *expr`, and the multiple
ways something can or can't be parsed like that.
Also makes yield-from a TODO in the bytecode generator.
Behold, the glory of javascript syntax:
```js
// 'yield' = expression in generators.
function* foo() {
yield
*bar; // <- Syntax error here, expression can't start with *
}
// 'yield' = identifier anywhere else.
function foo() {
yield
*bar; // Perfectly fine, this is just `yield * bar`
}
```
This is generated for Identifier nodes that represent a function
argument variable. It loads a given argument index from the current
call frame into the accumulator.
This counter is increased each time a synchronous execution sequence
completes, and will allow us to emulate the abstract operations
AddToKeptObjects & ClearKeptObjects efficiently.
This patch adds a CallType to the Bytecode::Op::Call instruction,
which can be either Call or Construct. We then generate Construct
calls for the NewExpression AST node.
When executed, these get fed into VM::construct().
This adds a new PushLexicalEnvironment instruction that creates a new
LexicalEnvironment and pushes it on the VM's scope stack.
There is no corresponding PopLexicalEnvironment instruction yet,
so this will behave incorrectly with let/const scopes for example.
This replaces Bytecode::Op::EnterScope with a new NewFunction op that
instantiates a ScriptFunction from a given FunctionNode (AST).
This is then used to instantiate the local functions directly from
bytecode when entering a ScopeNode. :^)
These will be partly handled by the relevant ScopeNode due to
hoisting, same basic idea as function declarations.
VariableDeclaration needs to do some work, but let's stub it out
first and start empty.
EnterUnwindContext pushes an unwind context (exception handler and/or
finalizer) onto a stack.
LeaveUnwindContext pops the unwind context from that stack.
Upon return to the interpreter loop we check whether the VM has an
exception pending. If no unwind context is available we return from the
loop. If an exception handler is available we clear the VM's exception,
put the exception value into the accumulator register, clear the unwind
context's handler and jump to the handler. If no handler is available
but a finalizer is available we save the exception value + metadata (for
later use by ContinuePendingUnwind), clear the VM's exception, pop the
unwind context and jump to the finalizer.
ContinuePendingUnwind checks whether a saved exception is available. If
no saved exception is available it jumps to the resume label. Otherwise
it stores the exception into the VM.
The Jump after LeaveUnwindContext could be integrated into the
LeaveUnwindContext instruction. I've kept them separate for now to make
the bytecode more readable.
> try { 1; throw "x" } catch (e) { 2 } finally { 3 }; 4
1:
[ 0] EnterScope
[ 10] EnterUnwindContext handler:@4 finalizer:@3
[ 38] EnterScope
[ 48] LoadImmediate 1
[ 60] NewString 1 ("x")
[ 70] Throw
<for non-terminated blocks: insert LeaveUnwindContext + Jump @3 here>
2:
[ 0] LoadImmediate 4
3:
[ 0] EnterScope
[ 10] LoadImmediate 3
[ 28] ContinuePendingUnwind resume:@2
4:
[ 0] SetVariable 0 (e)
[ 10] EnterScope
[ 20] LoadImmediate 2
[ 38] LeaveUnwindContext
[ 3c] Jump @3
String Table:
0: e
1: x
Instead of using Strings in the bytecode ops this adds a global string
table to the Executable struct which individual operations can refer
to using indices. This brings bytecode ops one step closer to being
pointer free.
Added Increment and Decrement bytecode ops to support this. Postfix
updates use a temporary register to preserve the original value.
Note that this patch only implements Identifier updates. Member
expression updates are a TODO.
This limits the size of each block (currently set to 1K), and gets us
closer to a canonical, more easily analysable bytecode format.
As a result of this, "Labels" are now simply entries to basic blocks.
Since there is no more 'conditional' jump (as all jumps are always
taken), JumpIf{True,False} are unified to JumpConditional, and
JumpIfNullish is renamed to JumpNullish.
Also fixes#7914 as a result of reimplementing the loop logic.
This change removes the mmap inside of Block in favor of a growing
vector of bytes. This is favorable for two reasons:
- We don't take more space than we need
- There is no limit to the growth of the vector (previously, if
the Block overstepped its 64kb boundary, it would just crash)
However, if that vector happens to resize, any pointer pointing into
that vector would become invalid. To avoid this, this commit adds an
InstructionHandle<Op> class which just stores a block and an offset
into that block.
This ensures that "while", do...while, "for" expressions have a
properly initialized result value even if the user terminated
the loop body via break or the loop body wasn't executed at all.
This partially reverts c6ce7c9326.
The munmap part of that change was good, but we can't seal the blocks
since that breaks NewString and other ops that have String members.
This commit introduces the concept of an accumulator register to
LibJS's bytecode interpreter. The accumulator register is always
register 0, and most simple instructions use it for reading and
writing.
Not only does this slim down the AST, but it also simplifies a lot of
the code. For example, the generate_bytecode methods no longer need
to return an Optional<Register>, as any opcode which has a "return"
value will always put it into the accumulator.
This also renames the old Op::Load to Op::LoadImmediate, and uses
Op::Load to load from a register into the accumulator. There is
also an Op::Store to put the value in the accumulator into another
register.
