This allows us to load modules from scripts.
This can be dangerous as it can load arbitrary files. Because of that it
fails and throws by default. Currently, only js and JavaScriptTestRunner
enable the default hook.
This also adds tests to test-js which test module code. Because we
form a spec perspective can't "enter" a module this is the easiest way
to run tests without having to modify test-js to have special cases for
modules.
To specify modules in test-js we use the extension '.mjs' this is to
ensure the files are not executed. We do still want to lint these files
so the prettier scripts have changed to look for '.mjs' files as well.
The big changes are:
- Allow strings as Module{Export, Import}Name
- Properly track declarations in default export statements
However, the spec is a little strange in that it allows function and
class declarations without a name in default export statements.
This is quite hard to fully implement without rewriting more of the
parser so for now this behavior is emulated by faking things with
function and class expressions. See the comments in
parse_export_statement for details on the hacks and where it goes wrong.
This also refactors interpreter creation to follow
InitializeHostDefinedRealm, but I couldn't fit it in the title :^)
This allows us to follow the spec much more closely rather than being
completely ad-hoc with just the parse node instead of having all the
surrounding data such as the realm of the parse node.
The interpreter creation refactor creates the global execution context
once and doesn't take it off the stack. This allows LibWeb to take the
global execution context and manually handle it, following the HTML
spec. The HTML spec calls this the "realm execution context" of the
environment settings object.
It also allows us to specify the globalThis type, as it can be
different from the global object type. For example, on the web, Window
global objects use a WindowProxy global this value to enforce the same
origin policy on operations like [[GetOwnProperty]].
Finally, it allows us to directly call Program::execute in perform_eval
and perform_shadow_realm_eval as this moves
global_declaration_instantiation into Interpreter::run
(ScriptEvaluation) as per the spec.
Note that this doesn't evalulate Source Text Modules yet or refactor
the bytecode interpreter, that's work for future us :^)
This patch was originally build by Luke for the environment settings
object change but was also needed for modules. So I (davidot) have
modified it with the new completion changes and setup for that.
Co-authored-by: davidot <davidot@serenityos.org>
Instead of using plain objects as Iterator records, causes confusion
about the object itself actually being its [[Iterator]] slot, and
requires non-standard type conversion shenanigans fpr the [[NextValue]]
and [[Done]] internal slots, implement a proper Iterator record struct
and use it throughout.
Also annotate the remaining Iterator AOs with spec comments while we're
here.
This includes:
- Parsing proper LabelledStatements with try_parse_labelled_statement()
- Removing LabelableStatement
- Implementing the LoopEvaluation semantics via loop_evaluation() in
each IterationStatement subclass; and IterationStatement evaluation
via {For,ForIn,ForOf,ForAwaitOf,While,DoWhile}Statement::execute()
- Updating ReturnStatement, BreakStatement and ContinueStatement to
return the appropriate completion types
- Basically reimplementing TryStatement and SwitchStatement according to
the spec, using completions
- Honoring result completion types in AsyncBlockStart and
OrdinaryCallEvaluateBody
- Removing any uses of the VM unwind mechanism - most importantly,
VM::throw_exception() now exclusively sets an exception and no longer
triggers any unwinding mechanism.
However, we already did a good job updating all of LibWeb and userland
applications to not use it, and the few remaining uses elsewhere don't
rely on unwinding AFAICT.
Instead of slapping 0..N labels on a statement like the current
LabelableStatement does, we need the spec's LabelledStatement for a
proper implementation of control flow using only completions - it always
has one label and one labelled statement - what appears to be 'multiple
labels' on the same statement is actually nested LabelledStatements.
Note that this is unused yet and will fully replace LabelableStatement
in the next commit.
This is another major milestone on our journey towards removing global
VM exception state :^)
Does pretty much exactly what it says on the tin: updating
ASTNode::execute() to return a Completion instead of a plain value. This
will *also* allow us to eventually remove the non-standard unwinding
mechanism and purely rely on the various completion types.
In the end this is a nicer API than having separate has_{value,target}()
and having to check those first, and then making another Optional from
the unwrapped value:
completion.has_value() ? completion.value() : Optional<Value> {}
// ^^^^^^^^^^^^^^^^^^
// Implicit creation of non-empty Optional<Value>
This way we need to unwrap the optional ourselves, but can easily pass
it to something else as well.
This is in anticipation of the AST using completions :^)
The spec has a note stating that resolve binding will always return a
reference whose [[ReferencedName]] field is name. However this is not
correct as the underlying method GetIdentifierReference may throw on
env.HasBinding(name) thus it can throw. However, there are some
scenarios where it cannot throw because the reference is known to exist
in that case we use MUST with a comment.
Exactly like in 99f9609, which fixed the same issue in CallExpression,
the spec tells us to *first* evaluate the arguments, if any, and *then*
check if the provided value is a constructor function.
Being really close to Object.prototype.valueOf() name wise makes this
unnecessarily confusing - while it sometimes serves as the
implementation of a valueOf() function, it's an abstraction which the
spec doesn't have.
Use the appropriate getters to retrieve specific internal slots instead,
most commonly [[FooData]] from the primitive wrapper objects.
For the Object class specifically, use the Value(Object*) ctor instead.
This commit adds support for the most bare bones version of async
functions, support for async generator functions, async arrow functions
and await expressions are TODO.
By replacing this VERIFY with a thrown Error we no longer crash when
calling a generator function in the AST interpreter. This allows us to
more gracefully handle situation which have not been implemented yet.
In particular this helps the libjs-test262-runner since it can now
continue on to the next tests instead of having the entire process end.
This one is a bit unusual, so to clarify:
Previously, callers of get_iterator_values() would supply a callback
that would return an IterationDecision, and an enum to indicate whether
iterator_close() should be invoked upon IterationDecision::Break.
Now, use of both those enums is removed, and callers must return an
Optional<Completion>. If a Completion is provided, the iterator will be
closed, and that completion will be returned from get_iterator_values.
Otherwise, once the iterator is exhausted, a default-initialized
Completion will be returned.
This is necessary as we might have to perform named evaluation with the
field name.
Ideally we would also skip some setup parts of the function like
function_declaration_instantiation however this would require bigger
changes to ECMAScriptFunctionObject.
This patch implements:
- Spec compliant [[Call]] and [[Construct]] internal slots, as virtual
FunctionObject::internal_{call,construct}(). These effectively replace
the old virtual FunctionObject::{call,construct}(), but with several
advantages:
- Clear and consistent naming, following the object internal methods
- Use of completions
- internal_construct() returns an Object, and not Value! This has been
a source of confusion for a long time, since in the spec there's
always an Object returned but the Value return type in LibJS meant
that this could not be fully trusted and something could screw you
over.
- Arguments are passed explicitly in form of a MarkedValueList,
allowing manipulation (BoundFunction). We still put them on the
execution context as a lot of code depends on it (VM::arguments()),
but not from the Call() / Construct() AOs anymore, which now allows
for bypassing them and invoking [[Call]] / [[Construct]] directly.
Nothing but Call() / Construct() themselves do that at the moment,
but future additions to ECMA262 or already existing web specs might.
- Spec compliant, standalone Call() and Construct() AOs: currently the
closest we have is VM::{call,construct}(), but those try to cater to
all the different function object subclasses at once, resulting in a
horrible mess and calling AOs with functions they should never be
called with; most prominently PrepareForOrdinaryCall and
OrdinaryCallBindThis, which are only for ECMAScriptFunctionObject.
As a result this also contains an implicit optimization: we no longer
need to create a new function environment for NativeFunctions - which,
worth mentioning, is what started this whole crusade in the first place
:^)
Most switch statements don't have any lexically scoped declarations,
so let's avoid allocating an environment in the common case where we
don't have to.
This commit partially reverts "LibJS: Make accessing the current
function's arguments cheaper".
While the change passed all the currently passing test262 tests, it
seems to have _some_ flaw that silently breaks with some real-world
websites.
As the speedup with negligible at best, let's just revert it until we
can implement it more correctly.
We now propagate this flag to FunctionDeclaration, and then also into
ECMAScriptFunctionObject.
This will be used to disable optimizations that aren't safe in the
presence of direct eval().
Instead of going through an environment record, make arguments of the
currently executing function generate references via the argument index,
which can later be resolved directly through the ExecutionContext.
This patch introduces the "environment coordinate" concept, which
encodes the distance from a variable access to the binding it ends up
resolving to.
EnvironmentCoordinate has two fields:
- hops: The number of hops up the lexical environment chain we have
to make before getting to the resolved binding.
- index: The index of the resolved binding within its declarative
environment record.
Whenever a variable lookup resolves somewhere inside a declarative
environment, we now cache the coordinates and reuse them in subsequent
lookups. This is achieved via a coordinate cache in JS::Identifier.
Note that non-strict direct eval() breaks this optimization and so it
will not be performed if the resolved environment has been permanently
screwed by eval().
This makes variable access *significantly* faster. :^)
The idea here is simple: If the block statement doesn't contain any
lexical declarations, we don't need to allocate, initialize and
eventually garbage collect a new declarative environment.
This even makes lookups across nested blocks slightly faster as we don't
have to traverse a chain of empty environments anymore - instead, the
execution context just stores the outermost non-empty one.
This doesn't speed up test-js considerably, but has a noticeable effect
on test262 and real-world web content :^)
This gives FunctionNode a "might need arguments object" boolean flag and
sets it based on the simplest possible heuristic for this: if we
encounter an identifier called "arguments" or "eval" up to the next
(nested) function declaration or expression, we won't need an arguments
object. Otherwise, we *might* need one - the final decision is made in
the FunctionDeclarationInstantiation AO.
Now, this is obviously not perfect. Even if you avoid eval, something
like `foo.arguments` will still trigger a false positive - but it's a
start and already massively cuts down on needlessly allocated objects,
especially in real-world code that is often minified, and so a full
"arguments" identifier will be an actual arguments object more often
than not.
To illustrate the actual impact of this change, here's the number of
allocated arguments objects during a full test-js run:
Before:
- Unmapped arguments objects: 78765
- Mapped arguments objects: 2455
After:
- Unmapped arguments objects: 18
- Mapped arguments objects: 37
This results in a ~5% speedup of test-js on my Linux host machine, and
about 3.5% on i686 Serenity in QEMU (warm runs, average of 5).
The following microbenchmark (calling an empty function 1M times) runs
25% faster on Linux and 45% on Serenity:
function foo() {}
for (var i = 0; i < 1_000_000; ++i)
foo();
test262 reports no changes in either direction, apart from a speedup :^)
Before this we used an ad-hoc combination of references and 'variables'
stored in a hashmap. This worked in most cases but is not spec like.
Additionally hoisting, dynamically naming functions and scope analysis
was not done properly.
This patch fixes all of that by:
- Implement BindingInitialization for destructuring assignment.
- Implementing a new ScopePusher which tracks the lexical and var
scoped declarations. This hoists functions to the top level if no
lexical declaration name overlaps. Furthermore we do checking of
redeclarations in the ScopePusher now requiring less checks all over
the place.
- Add methods for parsing the directives and statement lists instead
of having that code duplicated in multiple places. This allows
declarations to pushed to the appropriate scope more easily.
- Remove the non spec way of storing 'variables' in
DeclarativeEnvironment and make Reference follow the spec instead of
checking both the bindings and 'variables'.
- Remove all scoping related things from the Interpreter. And instead
use environments as specified by the spec. This also includes fixing
that NativeFunctions did not produce a valid FunctionEnvironment
which could cause issues with callbacks and eval. All
FunctionObjects now have a valid NewFunctionEnvironment
implementation.
- Remove execute_statements from Interpreter and instead use
ASTNode::execute everywhere this simplifies AST.cpp as you no longer
need to worry about which method to call.
- Make ScopeNodes setup their own environment. This uses four
different methods specified by the spec
{Block, Function, Eval, Global}DeclarationInstantiation with the
annexB extensions.
- Implement and use NamedEvaluation where specified.
Additionally there are fixes to things exposed by these changes to eval,
{for, for-in, for-of} loops and assignment.
Finally it also fixes some tests in test-js which where passing before
but not now that we have correct behavior :^).
Since there are only a number of statements where labels can actually be
used we now also only store labels when necessary.
Also now tracks the first continue usage of a label since this might not
be valid but that can only be determined after we have parsed the
statement.
Also ensures the correct error does not get wiped by load_state.
When we encounter a default clause in a switch statement, we should not
execute it immediately, instead we need to wait until all case clauses
have been executed as a matching case clause can break from the
switch/case.
The code is nowhere close to the spec, so instead of fixing it properly
I just made it slightly worse, but correct. Needs a complete refactor at
some point.
At a later point this will indicate whether some FunctionObject "has a
[[Construct]] internal method" (separate from the current FunctionObject
call() / construct()), to help with a more spec-compliant implementation
of [[Call]] and [[Construct]].
This means that it is no longer relevant to just NativeFunction.
