DeprecatedFlyString relies heavily on DeprecatedString's StringImpl, so
let's rename it to A) match the name of DeprecatedString, B) write a new
FlyString class that is tied to String.
These instances were detected by searching for files that include
AK/Format.h, but don't match the regex:
\\b(CheckedFormatString|critical_dmesgln|dbgln|dbgln_if|dmesgln|FormatBu
ilder|__FormatIfSupported|FormatIfSupported|FormatParser|FormatString|Fo
rmattable|Formatter|__format_value|HasFormatter|max_format_arguments|out
|outln|set_debug_enabled|StandardFormatter|TypeErasedFormatParams|TypeEr
asedParameter|VariadicFormatParams|v_critical_dmesgln|vdbgln|vdmesgln|vf
ormat|vout|warn|warnln|warnln_if)\\b
(Without the linebreaks.)
This regex is pessimistic, so there might be more files that don't
actually use any formatting functions.
Observe that this revealed that Userland/Libraries/LibC/signal.cpp is
missing an include.
In theory, one might use LibCPP to detect things like this
automatically, but let's do this one step after another.
Instead of CallExpression storing its arguments in a Vector<Argument>,
we now custom-allocate the memory slot for CallExpression (and its
subclass NewExpression) so that it fits both CallExpression and its list
of Arguments in one allocation.
This reduces memory usage on twitter.com/awesomekling by 8.8 MiB :^)
This is still not perfect, as we now actually crash in the
`try-finally-continue` tests, while we now succeed all
`try-catch-finally-*` tests.
Note that we do not yet go through the finally block when exiting the
unwind context through a break or continue.
Before we were doing so while exiting the catch-block, but not when
exiting the try-block.
This now centralizes the responsibility to exit the unwind context to
the finalizer, ignoring return/break/continue.
This makes it easier to handle the return case in a future commit.
We have a new, improved string type coming up in AK (OOM aware, no null
state), and while it's going to use UTF-8, the name UTF8String is a
mouthful - so let's free up the String name by renaming the existing
class.
Making the old one have an annoying name will hopefully also help with
quick adoption :^)
Previously, throw and return completions would not be executed inside
the generator. This is incorrect, as throw and return need to perform
unwinds which can potentially execute more code inside the generator,
such as finally blocks.
This is done by also passing the completion type alongside the passed
in value. The continuation block will immediately extract and type and
value and perform the appropriate operation for the given type.
For normal completions, this is continuing as normal.
For throw completions, it will perform `throw <value>`.
For return completions, it will perform `return <value>`, which is a
`Yield return` in this case due to being inside a generator.
This also refactors GeneratorObject to properly send across the
completion type and value to the generator inside of trying to operate
on the completions itself.
This is a prerequisite for yield*, as it performs special iterator
operations when receiving a throw/return completion and does not
complete the generator like the regular yield would.
There's still more work to be done to make GeneratorObject::execute
be closer to the spec. It's mostly a restructuring of the existing
GeneratorObject::next_impl.
We were mistakenly treating these as `for (x of obj)`. By reorganizing
the code a little bit, we actually support both kinds of iteration with
less duplication. :^)
Fixes 17 tests in test262.
This gives us better debug output when analysing calls to `undefined`
and also fixes multiple test-js cases expecting an
`(evaluated from $Expression)` in the error message.
This also refactors out the generation of that string, to avoid code
duplication with the AST interpreter.
This is no longer required, since the variable scope is ended after
switching to the end block, which means that LeaveLexicalEnvironment
will always be generated instead of depending on the unwind mechanism
to handle it for us.
BlockDeclarationInstantiation takes as input the new lexical
environment that was created and checks if there is a binding for the
current name only in this new scope.
This allows shadowing lexical variables and prevents us crashing due to
an already initialized lexical variable in this case:
```js
let x = 1;
{
let x = 1;
}
```
If the for loop's body is not block terminated, we will generate a Jump
to the end block which will block terminate the body. Then, we ended
the lexical variable scope if needed. However, since the body is now
block terminated, the "LeaveLexicalEnvironment" instruction that is
generated by end_variable_scope is now dropped on the floor.
This fixes this by moving it to the beginning of the end block.
Previously we only did this if the body block was not terminated.
If it was, all future codegen would happen in this block terminated
body block until another switch occurred, dropping all generated
instructions in this time on the floor.
This allows you to recurse into a named function that is stored in a
variable. For example, this would previously print "wrong" instead of
"right":
```js
function g() { console.log("wrong") }
f = function g(i) { if (i !== 1) g(1); else console.log("right"); }
f()
```
This is done by keeping track of all the labels that apply to a given
break/continue scope alongside their bytecode target. When a
break/continue with a label is generated, we scan from the most inner
scope to the most outer scope looking for the label, performing any
necessary unwinds on the way. Once the label is found, it is then
jumped to.
`delete` has to operate directly on Reference Records, so this
introduces a new set of operations called DeleteByValue, DeleteVariable
and DeleteById. They operate similarly to their Get counterparts,
except they end in creating a (temporary) Reference and calling delete_
on it.
The body of for/in/of can contain an unconditional block terminator
(e.g. return, throw), so we have to check for that before generating
the Jump to the loop update block.
In object binding, we would attempt to get NonnullRefPtr<Identifier>
from alias on the alias.has<Empty>() code path. In this case, we need
to get it from name instead.
The update block can generate bytecode that refers to the lexical
environment, so we have to end the scope after it has been generated.
Previously the Jump to the update block would terminate the block,
causing us to leave the lexical environment just before jumping to the
update block.
After we terminate a block (e.g. break, continue), we cannot generate
anymore bytecode for the block. This caused us to crash with this
example code:
```
a = 0;
switch (a) {
case 0:
break;
console.log("hello world");
}
```
Anything after a block terminating instruction is considered
unreachable code, so we can safely skip any statements after it.
Listing all the registers will lead to the inability to allocate enough
space in one basic block (as there can be an arbitrary number of
registers used), instead switch to specifying the range of registers
used and save a lot of space in the process.
This will leave any lexical/variable environments on the way to the
closest unwind context boundary.
This will not leave the closest unwind context, as we still need the
unwind context to perform the Throw instruction correctly.
Previously we would only end these scopes if the block was not
terminated. If the block was generated, we would not end the scope
and would generate other bytecode with these scopes still open.
These functions do not generate any code, so they can be used even if
the current block is terminated. The enter and end scope functions are
only used to track where to unwind to when break/continue are used.
Now we emit CreateVariable and SetVariable with the appropriate
initialization/environment modes, much closer to the spec.
This makes a whole lot of things like let/const variables, function
and variable hoisting and some other things work :^)
Instead of crashing on the spot, return a descriptive error that will
eventually continue its days as a javascript "InternalError" exception.
This should make random crashes with BC less likely.
This removes a number of vm.exception() checks which are now caught
directly by TRY. Make use of these checks in
{Global, Eval}DeclarationInstantiation and while we're here add spec
comments.
This is a specialized string table for storing identifiers only.
Identifiers are always FlyStrings, which makes many common operations
faster by allowing O(1) comparison.
Before this we used an ad-hoc combination of references and 'variables'
stored in a hashmap. This worked in most cases but is not spec like.
Additionally hoisting, dynamically naming functions and scope analysis
was not done properly.
This patch fixes all of that by:
- Implement BindingInitialization for destructuring assignment.
- Implementing a new ScopePusher which tracks the lexical and var
scoped declarations. This hoists functions to the top level if no
lexical declaration name overlaps. Furthermore we do checking of
redeclarations in the ScopePusher now requiring less checks all over
the place.
- Add methods for parsing the directives and statement lists instead
of having that code duplicated in multiple places. This allows
declarations to pushed to the appropriate scope more easily.
- Remove the non spec way of storing 'variables' in
DeclarativeEnvironment and make Reference follow the spec instead of
checking both the bindings and 'variables'.
- Remove all scoping related things from the Interpreter. And instead
use environments as specified by the spec. This also includes fixing
that NativeFunctions did not produce a valid FunctionEnvironment
which could cause issues with callbacks and eval. All
FunctionObjects now have a valid NewFunctionEnvironment
implementation.
- Remove execute_statements from Interpreter and instead use
ASTNode::execute everywhere this simplifies AST.cpp as you no longer
need to worry about which method to call.
- Make ScopeNodes setup their own environment. This uses four
different methods specified by the spec
{Block, Function, Eval, Global}DeclarationInstantiation with the
annexB extensions.
- Implement and use NamedEvaluation where specified.
Additionally there are fixes to things exposed by these changes to eval,
{for, for-in, for-of} loops and assignment.
Finally it also fixes some tests in test-js which where passing before
but not now that we have correct behavior :^).
The parser doesn't always track lexical scopes correctly, so let's not
rely on that for direct argument loading.
This reverts the LoadArguments bytecode instruction as well. We can
bring these things back when the parser can reliably tell us that
a given Identifier is indeed a function argument.
This patch makes the following name changes:
- ScopeObject => EnvironmentRecord
- LexicalEnvironment => DeclarativeEnvironmentRecord
- WithScope => ObjectEnvironmentRecord
This commit implements parsing for `yield *expr`, and the multiple
ways something can or can't be parsed like that.
Also makes yield-from a TODO in the bytecode generator.
Behold, the glory of javascript syntax:
```js
// 'yield' = expression in generators.
function* foo() {
yield
*bar; // <- Syntax error here, expression can't start with *
}
// 'yield' = identifier anywhere else.
function foo() {
yield
*bar; // Perfectly fine, this is just `yield * bar`
}
```
This is generated for Identifier nodes that represent a function
argument variable. It loads a given argument index from the current
call frame into the accumulator.
This patch adds a CallType to the Bytecode::Op::Call instruction,
which can be either Call or Construct. We then generate Construct
calls for the NewExpression AST node.
When executed, these get fed into VM::construct().
This adds a new PushLexicalEnvironment instruction that creates a new
LexicalEnvironment and pushes it on the VM's scope stack.
There is no corresponding PopLexicalEnvironment instruction yet,
so this will behave incorrectly with let/const scopes for example.
This replaces Bytecode::Op::EnterScope with a new NewFunction op that
instantiates a ScriptFunction from a given FunctionNode (AST).
This is then used to instantiate the local functions directly from
bytecode when entering a ScopeNode. :^)
These will be partly handled by the relevant ScopeNode due to
hoisting, same basic idea as function declarations.
VariableDeclaration needs to do some work, but let's stub it out
first and start empty.
EnterUnwindContext pushes an unwind context (exception handler and/or
finalizer) onto a stack.
LeaveUnwindContext pops the unwind context from that stack.
Upon return to the interpreter loop we check whether the VM has an
exception pending. If no unwind context is available we return from the
loop. If an exception handler is available we clear the VM's exception,
put the exception value into the accumulator register, clear the unwind
context's handler and jump to the handler. If no handler is available
but a finalizer is available we save the exception value + metadata (for
later use by ContinuePendingUnwind), clear the VM's exception, pop the
unwind context and jump to the finalizer.
ContinuePendingUnwind checks whether a saved exception is available. If
no saved exception is available it jumps to the resume label. Otherwise
it stores the exception into the VM.
The Jump after LeaveUnwindContext could be integrated into the
LeaveUnwindContext instruction. I've kept them separate for now to make
the bytecode more readable.
> try { 1; throw "x" } catch (e) { 2 } finally { 3 }; 4
1:
[ 0] EnterScope
[ 10] EnterUnwindContext handler:@4 finalizer:@3
[ 38] EnterScope
[ 48] LoadImmediate 1
[ 60] NewString 1 ("x")
[ 70] Throw
<for non-terminated blocks: insert LeaveUnwindContext + Jump @3 here>
2:
[ 0] LoadImmediate 4
3:
[ 0] EnterScope
[ 10] LoadImmediate 3
[ 28] ContinuePendingUnwind resume:@2
4:
[ 0] SetVariable 0 (e)
[ 10] EnterScope
[ 20] LoadImmediate 2
[ 38] LeaveUnwindContext
[ 3c] Jump @3
String Table:
0: e
1: x
Instead of using Strings in the bytecode ops this adds a global string
table to the Executable struct which individual operations can refer
to using indices. This brings bytecode ops one step closer to being
pointer free.
Added Increment and Decrement bytecode ops to support this. Postfix
updates use a temporary register to preserve the original value.
Note that this patch only implements Identifier updates. Member
expression updates are a TODO.
This limits the size of each block (currently set to 1K), and gets us
closer to a canonical, more easily analysable bytecode format.
As a result of this, "Labels" are now simply entries to basic blocks.
Since there is no more 'conditional' jump (as all jumps are always
taken), JumpIf{True,False} are unified to JumpConditional, and
JumpIfNullish is renamed to JumpNullish.
Also fixes#7914 as a result of reimplementing the loop logic.
This change removes the mmap inside of Block in favor of a growing
vector of bytes. This is favorable for two reasons:
- We don't take more space than we need
- There is no limit to the growth of the vector (previously, if
the Block overstepped its 64kb boundary, it would just crash)
However, if that vector happens to resize, any pointer pointing into
that vector would become invalid. To avoid this, this commit adds an
InstructionHandle<Op> class which just stores a block and an offset
into that block.
This ensures that "while", do...while, "for" expressions have a
properly initialized result value even if the user terminated
the loop body via break or the loop body wasn't executed at all.
This commit introduces the concept of an accumulator register to
LibJS's bytecode interpreter. The accumulator register is always
register 0, and most simple instructions use it for reading and
writing.
Not only does this slim down the AST, but it also simplifies a lot of
the code. For example, the generate_bytecode methods no longer need
to return an Optional<Register>, as any opcode which has a "return"
value will always put it into the accumulator.
This also renames the old Op::Load to Op::LoadImmediate, and uses
Op::Load to load from a register into the accumulator. There is
also an Op::Store to put the value in the accumulator into another
register.
This patch changes the LibJS bytecode to be a stream of instructions
packed one-after-the-other in contiguous memory, instead of a vector
of OwnPtr<Instruction>. This should be a lot more cache-friendly. :^)
Instructions are also devirtualized and instead have a type field
using a new Instruction::Type enum.
To iterate over a bytecode stream, one must now use
Bytecode::InstructionStreamIterator.
Unlike the convoluted unwind-until-scope-type mechanism in the AST
interpreter, "continue" maps to a simple Bytecode::Op::Jump here. :^)
We know where to jump based on a stack of "continuable scopes" that
we now maintain on the Bytecode::Generator as we go.
Note that this only supports bare "continue", not continue-with-label.
This also required making Bytecode::Op::Jump support lazy linking
to a target label.
I left a FIXME here about having the "if" statement return the result
value from the taken branch statement. That's what the AST interpreter
does but I'm not sure if it's actually required.
If there's a current Bytecode::Interpreter in action, ScriptFunction
will now compile itself into bytecode and execute in that context.
This patch also adds the Return bytecode instruction so that we can
actually return values from called functions. :^)
Return values are propagated from callee to caller via the caller's
$0 register. Bytecode::Interpreter now keeps a stack of register
"windows". These are not very efficient, but it should be pretty
straightforward to convert them to e.g a sliding register window
architecture later on.
This is pretty dang cool! :^)
This patch adds the Call bytecode instruction which is emitted for the
CallExpression AST node.
It's pretty barebones and doesn't handle 'this' values properly, etc.
But it can perform basic function calls! :^)
Note that the called function will *not* execute as bytecode, but will
simply fall back into the old codepath and use the AST interpreter.
This is intended to perform the same duties as enter_scope() does in
the AST tree-walk interpreter:
- Hoisted function declaration processing
- Hoisted variable declaration processing
- ... maybe more
This first cut only implements the function declaration processing.
This was quite straightforward using the same label/jump machinery that
we added for while statements.
The main addition here is a new JumpIfTrue bytecode instruction.
This introduces two new instructions: Jump and JumpIfFalse.
Jumps are made to a Bytecode::Label, which is a simple object that
represents a location in the bytecode stream.
Note that you may not always know the target of a jump when adding the
jump instruction itself, but we can just update the instruction later
on during codegen once we know where the jump target is.
The Bytecode::Interpreter now implements jumping via a jump slot that
gets checked after each instruction to see if a jump is pending.
If not, we just increment the PC as usual.
