This is where the spec wants to have it. Requires a couple of hacks as
currently everything that needs a Realm actually has a GlobalObject, so
we need to go via the Interpreter.
The test262 tests under RegExp/property-escapes/generated will invoke
Reflect.apply with up to 10,000 arguments at a time. In LibJS, when the
call stack reached VM::call_internal, we transfer those arguments from
a MarkedValueList to the execution context's arguments Vector.
Because these types differ (MarkedValueList is a Vector<Value, 32>), the
arguments are copied rather than moved. By changing the arguments vector
to a MarkedValueList, we can properly move the passed arguments over.
This shaves about 2 seconds off the following test262 test (from 15sec):
RegExp/property-escapes/generated/General_Category_-_Decimal_Number.js
RegExpInitialize specifies how the pattern string should be created
before passing it to [[RegExpMatcher]]. Rather than passing it as-is,
the string should be converted to code points and back to a "List" (if
the Unicode flag is present), or as a "List" of UTF-16 code units.
Further. the spec requires that we keep both the original pattern string
and this parsed string in the RegExp object.
The caveat is that the LibRegex parser further requires any multi-byte
code units to be escaped (as "\unnnn"). Otherwise, the code unit is
recognized as individual UTF-8 bytes.
This removes all usages of the non-standard put helper method and
replaces all of it's usages with the specification required alternative
or with define_direct_property where appropriate.
This removes all usages of the non-standard define_property helper
method and replaces all it's usages with the specification required
alternative or with define_direct_property where appropriate.
This is a huge patch, I know. In hindsight this perhaps could've been
done slightly more incremental, but I started and then fixed everything
until it worked, and here we are. I tried splitting of some completely
unrelated changes into separate commits, however. Anyway.
This is a rewrite of most of Object, and by extension large parts of
Array, Proxy, Reflect, String, TypedArray, and some other things.
What we already had worked fine for about 90% of things, but getting the
last 10% right proved to be increasingly difficult with the current code
that sort of grew organically and is only very loosely based on the
spec - this became especially obvious when we started fixing a large
number of test262 failures.
Key changes include:
- 1:1 matching function names and parameters of all object-related
functions, to avoid ambiguity. Previously we had things like put(),
which the spec doesn't have - as a result it wasn't always clear which
need to be used.
- Better separation between object abstract operations and internal
methods - the former are always the same, the latter can be overridden
(and are therefore virtual). The internal methods (i.e. [[Foo]] in the
spec) are now prefixed with 'internal_' for clarity - again, it was
previously not always clear which AO a certain method represents,
get() could've been both Get and [[Get]] (I don't know which one it
was closer to right now).
Note that some of the old names have been kept until all code relying
on them is updated, but they are now simple wrappers around the
closest matching standard abstract operation.
- Simplifications of the storage layer: functions that write values to
storage are now prefixed with 'storage_' to make their purpose clear,
and as they are not part of the spec they should not contain any steps
specified by it. Much functionality is now covered by the layers above
it and was removed (e.g. handling of accessors, attribute checks).
- PropertyAttributes has been greatly simplified, and is being replaced
by PropertyDescriptor - a concept similar to the current
implementation, but more aligned with the actual spec. See the commit
message of the previous commit where it was introduced for details.
- As a bonus, and since I had to look at the spec a whole lot anyway, I
introduced more inline comments with the exact steps from the spec -
this makes it super easy to verify correctness.
- East-const all the things.
As a result of all of this, things are much more correct but a bit
slower now. Retaining speed wasn't a consideration at all, I have done
no profiling of the new code - there might be low hanging fruits, which
we can then harvest separately.
Special thanks to Idan for helping me with this by tracking down bugs,
updating everything outside of LibJS to work with these changes (LibWeb,
Spreadsheet, HackStudio), as well as providing countless patches to fix
regressions I introduced - there still are very few (we got it down to
5), but we also get many new passing test262 tests in return. :^)
Co-authored-by: Idan Horowitz <idan.horowitz@gmail.com>
Specifically, this now explicitly takes the length, adds missing
exceptions checks to calls with user-supplied lengths, takes and uses
the prototype argument, and fixes some spec non-conformance in
ArrayConstructor and its native functions around the use of ArrayCreate
The parser doesn't always track lexical scopes correctly, so let's not
rely on that for direct argument loading.
This reverts the LoadArguments bytecode instruction as well. We can
bring these things back when the parser can reliably tell us that
a given Identifier is indeed a function argument.
These represent the outermost scope in the environment record
hierarchy. The spec says they should be a "composite" of two things:
- An ObjectEnvironmentRecord wrapping the global object
- A DeclarativeEnvironmentRecord for other declarations
It's not yet clear to me how this should work, so this patch only
implements the first part, an object record wrapping the global object.
To better follow the spec, we need to distinguish between the current
execution context's lexical environment and variable environment.
This patch moves us to having two record pointers, although both of
them point at the same environment records for now.
This patch makes the following name changes:
- ScopeObject => EnvironmentRecord
- LexicalEnvironment => DeclarativeEnvironmentRecord
- WithScope => ObjectEnvironmentRecord
This now matches the spec's OrdinaryObjectCreate() across the board:
instead of implicitly setting the created object's prototype to
%Object.prototype% and then in many cases setting it to a nullptr right
away, it now has an 'Object* prototype' parameter with _no default
value_. This makes the code easier to compare with the spec, very clear
in terms of what prototype is being used as well as avoiding unnecessary
shape transitions.
Also fixes a couple of cases were we weren't setting the correct
prototype.
There's no reason to assume that the object would not be empty (as in
having own properties), so let's follow our existing pattern of
Type::create(...) and simply call it 'create'.
This commit adds a bunch of passes, the most interesting of which is a
pass that merges blocks together, and a pass that places blocks that
flow into each other next to each other, and a very simply pass that
removes duplicate basic blocks.
Note that this does not remove the jump at the end of each block in that
pass to avoid scope creep in the passes.
These are pretty hairy if someone forgets to override one, as the
catchall function in Instruction will keep calling itself over and over
again, leading to really hard-to-debug situations.
This commit implements parsing for `yield *expr`, and the multiple
ways something can or can't be parsed like that.
Also makes yield-from a TODO in the bytecode generator.
Behold, the glory of javascript syntax:
```js
// 'yield' = expression in generators.
function* foo() {
yield
*bar; // <- Syntax error here, expression can't start with *
}
// 'yield' = identifier anywhere else.
function foo() {
yield
*bar; // Perfectly fine, this is just `yield * bar`
}
```
This is generated for Identifier nodes that represent a function
argument variable. It loads a given argument index from the current
call frame into the accumulator.
This counter is increased each time a synchronous execution sequence
completes, and will allow us to emulate the abstract operations
AddToKeptObjects & ClearKeptObjects efficiently.
This patch adds a CallType to the Bytecode::Op::Call instruction,
which can be either Call or Construct. We then generate Construct
calls for the NewExpression AST node.
When executed, these get fed into VM::construct().
This adds a new PushLexicalEnvironment instruction that creates a new
LexicalEnvironment and pushes it on the VM's scope stack.
There is no corresponding PopLexicalEnvironment instruction yet,
so this will behave incorrectly with let/const scopes for example.
This replaces Bytecode::Op::EnterScope with a new NewFunction op that
instantiates a ScriptFunction from a given FunctionNode (AST).
This is then used to instantiate the local functions directly from
bytecode when entering a ScopeNode. :^)
These will be partly handled by the relevant ScopeNode due to
hoisting, same basic idea as function declarations.
VariableDeclaration needs to do some work, but let's stub it out
first and start empty.
EnterUnwindContext pushes an unwind context (exception handler and/or
finalizer) onto a stack.
LeaveUnwindContext pops the unwind context from that stack.
Upon return to the interpreter loop we check whether the VM has an
exception pending. If no unwind context is available we return from the
loop. If an exception handler is available we clear the VM's exception,
put the exception value into the accumulator register, clear the unwind
context's handler and jump to the handler. If no handler is available
but a finalizer is available we save the exception value + metadata (for
later use by ContinuePendingUnwind), clear the VM's exception, pop the
unwind context and jump to the finalizer.
ContinuePendingUnwind checks whether a saved exception is available. If
no saved exception is available it jumps to the resume label. Otherwise
it stores the exception into the VM.
The Jump after LeaveUnwindContext could be integrated into the
LeaveUnwindContext instruction. I've kept them separate for now to make
the bytecode more readable.
> try { 1; throw "x" } catch (e) { 2 } finally { 3 }; 4
1:
[ 0] EnterScope
[ 10] EnterUnwindContext handler:@4 finalizer:@3
[ 38] EnterScope
[ 48] LoadImmediate 1
[ 60] NewString 1 ("x")
[ 70] Throw
<for non-terminated blocks: insert LeaveUnwindContext + Jump @3 here>
2:
[ 0] LoadImmediate 4
3:
[ 0] EnterScope
[ 10] LoadImmediate 3
[ 28] ContinuePendingUnwind resume:@2
4:
[ 0] SetVariable 0 (e)
[ 10] EnterScope
[ 20] LoadImmediate 2
[ 38] LeaveUnwindContext
[ 3c] Jump @3
String Table:
0: e
1: x
Instead of using Strings in the bytecode ops this adds a global string
table to the Executable struct which individual operations can refer
to using indices. This brings bytecode ops one step closer to being
pointer free.
Added Increment and Decrement bytecode ops to support this. Postfix
updates use a temporary register to preserve the original value.
Note that this patch only implements Identifier updates. Member
expression updates are a TODO.
This limits the size of each block (currently set to 1K), and gets us
closer to a canonical, more easily analysable bytecode format.
As a result of this, "Labels" are now simply entries to basic blocks.
Since there is no more 'conditional' jump (as all jumps are always
taken), JumpIf{True,False} are unified to JumpConditional, and
JumpIfNullish is renamed to JumpNullish.
Also fixes#7914 as a result of reimplementing the loop logic.
This change removes the mmap inside of Block in favor of a growing
vector of bytes. This is favorable for two reasons:
- We don't take more space than we need
- There is no limit to the growth of the vector (previously, if
the Block overstepped its 64kb boundary, it would just crash)
However, if that vector happens to resize, any pointer pointing into
that vector would become invalid. To avoid this, this commit adds an
InstructionHandle<Op> class which just stores a block and an offset
into that block.
This ensures that "while", do...while, "for" expressions have a
properly initialized result value even if the user terminated
the loop body via break or the loop body wasn't executed at all.
This partially reverts c6ce7c9326.
The munmap part of that change was good, but we can't seal the blocks
since that breaks NewString and other ops that have String members.
This commit introduces the concept of an accumulator register to
LibJS's bytecode interpreter. The accumulator register is always
register 0, and most simple instructions use it for reading and
writing.
Not only does this slim down the AST, but it also simplifies a lot of
the code. For example, the generate_bytecode methods no longer need
to return an Optional<Register>, as any opcode which has a "return"
value will always put it into the accumulator.
This also renames the old Op::Load to Op::LoadImmediate, and uses
Op::Load to load from a register into the accumulator. There is
also an Op::Store to put the value in the accumulator into another
register.
After compiling bytecode, we should mark the memory read-only.
This currently does not work because it breaks instruction destruction.
I'm adding this anyway with a FIXME so we don't forget about it. :^)
This patch changes the LibJS bytecode to be a stream of instructions
packed one-after-the-other in contiguous memory, instead of a vector
of OwnPtr<Instruction>. This should be a lot more cache-friendly. :^)
Instructions are also devirtualized and instead have a type field
using a new Instruction::Type enum.
To iterate over a bytecode stream, one must now use
Bytecode::InstructionStreamIterator.
The Bytecode::Interpreter will push a global call frame if needed,
and it needs to make sure that call frame survives until the end
of the Interpreter::run() function.
Unlike the convoluted unwind-until-scope-type mechanism in the AST
interpreter, "continue" maps to a simple Bytecode::Op::Jump here. :^)
We know where to jump based on a stack of "continuable scopes" that
we now maintain on the Bytecode::Generator as we go.
Note that this only supports bare "continue", not continue-with-label.
This also required making Bytecode::Op::Jump support lazy linking
to a target label.
I left a FIXME here about having the "if" statement return the result
value from the taken branch statement. That's what the AST interpreter
does but I'm not sure if it's actually required.
If there's a current Bytecode::Interpreter in action, ScriptFunction
will now compile itself into bytecode and execute in that context.
This patch also adds the Return bytecode instruction so that we can
actually return values from called functions. :^)
Return values are propagated from callee to caller via the caller's
$0 register. Bytecode::Interpreter now keeps a stack of register
"windows". These are not very efficient, but it should be pretty
straightforward to convert them to e.g a sliding register window
architecture later on.
This is pretty dang cool! :^)
This patch adds the Call bytecode instruction which is emitted for the
CallExpression AST node.
It's pretty barebones and doesn't handle 'this' values properly, etc.
But it can perform basic function calls! :^)
Note that the called function will *not* execute as bytecode, but will
simply fall back into the old codepath and use the AST interpreter.
This is intended to perform the same duties as enter_scope() does in
the AST tree-walk interpreter:
- Hoisted function declaration processing
- Hoisted variable declaration processing
- ... maybe more
This first cut only implements the function declaration processing.
This was quite straightforward using the same label/jump machinery that
we added for while statements.
The main addition here is a new JumpIfTrue bytecode instruction.
This introduces two new instructions: Jump and JumpIfFalse.
Jumps are made to a Bytecode::Label, which is a simple object that
represents a location in the bytecode stream.
Note that you may not always know the target of a jump when adding the
jump instruction itself, but we can just update the instruction later
on during codegen once we know where the jump target is.
The Bytecode::Interpreter now implements jumping via a jump slot that
gets checked after each instruction to see if a jump is pending.
If not, we just increment the PC as usual.
- NewString (allocates a new PrimitiveString from the GC heap)
- GetVariable (retrieves a variable in the current scope)
- SetVariable (assigns a variable in the current scope)
This patch begins the work of implementing JavaScript execution in a
bytecode VM instead of an AST tree-walk interpreter.
It's probably quite naive, but we have to start somewhere.
The basic idea is that you call Bytecode::Generator::generate() on an
AST node and it hands you back a Bytecode::Block filled with
instructions that can then be interpreted by a Bytecode::Interpreter.
This first version only implements two instructions: Load and Add. :^)
Each bytecode block has infinity registers, and the interpreter resizes
its register file to fit the block being executed.
Two new `js` options are added in this patch as well:
`-d` will dump the generated bytecode
`-b` will execute the generated bytecode
Note that unless `-d` and/or `-b` are specified, none of the bytecode
related stuff in LibJS runs at all. This is implemented in parallel
with the existing AST interpreter. :^)
