While adding spec comments to PerformEval, I noticed we were missing
multiple steps.
Namely, these were:
- Checking if the host will allow us to compile the string
(allowing LibWeb to perform CSP for eval)
- The parser's initial state depending on the environment around us
on direct eval:
- Allowing new.target via eval in functions
- Allowing super calls and super properties via eval in classes
- Disallowing the use of the arguments object in class field
initializers at eval's parse time
- Setting ScriptOrModule of eval's execution context
The spec allows us to apply the additional parsing steps in any order.
The method I have gone with is passing in a struct to the parser's
constructor, which overrides the parser's initial state to (dis)allow
the things stated above from the get-go.
The spec version of canonical_numeric_index_string is absurdly complex,
and ends up converting from a string to a number, and then back again
which is both slow and also requires a few allocations and a string
compare.
Instead this patch moves away from using Values to represent canonical
a canonical index. In most cases all we need to know is whether a
PropertyKey is an integer between 0 and 2^^32-2, which we already
compute when we construct a PropertyKey so the existing is_number()
check is sufficient.
The more expensive case is handling strings containing numbers that
don't roundtrip through string conversion. In most cases these turn
into regular string properties, but for TypedArray access these
property names are not treated as normal named properties.
TypedArrays treat these numeric properties as magic indexes that are
ignored on read and are not stored (but are evaluated) on assignment.
For that reason there's now a mode flag on canonical_numeric_index_string
so that only TypedArrays take the cost of the ToString round trip test.
In order to improve the performance of this path this patch includes
some early returns to avoid conversion in cases where we can quickly
know whether a property can round trip.
This reverts commit 3a184f7841.
This broke a number of test262 tests under "TypedArrayConstructors".
The issue is that the CanonicalNumericIndexString AO should not fail
for inputs like "1.1", despite them not being integral indices.
Instead of crashing on the spot, return a descriptive error that will
eventually continue its days as a javascript "InternalError" exception.
This should make random crashes with BC less likely.
The spec version of canonical_numeric_index_string is absurdly complex,
and ends up converting from a string to a number, and then back again
which is both slow and also requires a few allocations and a string
compare.
Instead lets use the logic we already have as that is much more
efficient.
This improves performance of all non-numeric property names.
This commit removes all exception related code:
Remove VM::exception(), VM::throw_exception() etc. Any leftover
throw_exception calls are moved to throw_completion.
The one method left is clear_exception() which is now a no-op. Most of
these calls are just to clear whatever exception might have been thrown
when handling a Completion. So to have a cleaner commit this will be
removed in a next commit.
It also removes the actual Exception and TemporaryClearException classes
since these are no longer used.
In any spot where the exception was actually used an attempt was made to
preserve that behavior. However since it is no longer tracked by the VM
we cannot access exceptions which were thrown in previous calls.
There are two such cases which might have different behavior:
- In Web::DOM::Document::interpreter() the on_call_stack_emptied hook
used to print any uncaught exception but this is now no longer
possible as the VM does not store uncaught exceptions.
- In js the code used to be interruptable by throwing an exception on
the VM. This is no longer possible but was already somewhat fragile
before as you could happen to throw an exception just before a VERIFY.
This removes a number of vm.exception() checks which are now caught
directly by TRY. Make use of these checks in
{Global, Eval}DeclarationInstantiation and while we're here add spec
comments.
Using an Optional was extremely wasteful for function objects that don't
even have a bytecode executable.
This allows ECMAScriptFunctionObject to fit in a smaller size class.
This also refactors interpreter creation to follow
InitializeHostDefinedRealm, but I couldn't fit it in the title :^)
This allows us to follow the spec much more closely rather than being
completely ad-hoc with just the parse node instead of having all the
surrounding data such as the realm of the parse node.
The interpreter creation refactor creates the global execution context
once and doesn't take it off the stack. This allows LibWeb to take the
global execution context and manually handle it, following the HTML
spec. The HTML spec calls this the "realm execution context" of the
environment settings object.
It also allows us to specify the globalThis type, as it can be
different from the global object type. For example, on the web, Window
global objects use a WindowProxy global this value to enforce the same
origin policy on operations like [[GetOwnProperty]].
Finally, it allows us to directly call Program::execute in perform_eval
and perform_shadow_realm_eval as this moves
global_declaration_instantiation into Interpreter::run
(ScriptEvaluation) as per the spec.
Note that this doesn't evalulate Source Text Modules yet or refactor
the bytecode interpreter, that's work for future us :^)
This patch was originally build by Luke for the environment settings
object change but was also needed for modules. So I (davidot) have
modified it with the new completion changes and setup for that.
Co-authored-by: davidot <davidot@serenityos.org>
This is another major milestone on our journey towards removing global
VM exception state :^)
Does pretty much exactly what it says on the tin: updating
ASTNode::execute() to return a Completion instead of a plain value. This
will *also* allow us to eventually remove the non-standard unwinding
mechanism and purely rely on the various completion types.
The spec has a note stating that resolve binding will always return a
reference whose [[ReferencedName]] field is name. However this is not
correct as the underlying method GetIdentifierReference may throw on
env.HasBinding(name) thus it can throw. However, there are some
scenarios where it cannot throw because the reference is known to exist
in that case we use MUST with a comment.
Now that only ECMAScriptFunctionObject uses this, we can remove the
FunctionObject::new_function_environment() pure virtual method and just
implement it as a standalone AO with an ECMAScriptFunctionObject
parameter, next to the other NewFooEnvironment AOs.
This patch implements:
- Spec compliant [[Call]] and [[Construct]] internal slots, as virtual
FunctionObject::internal_{call,construct}(). These effectively replace
the old virtual FunctionObject::{call,construct}(), but with several
advantages:
- Clear and consistent naming, following the object internal methods
- Use of completions
- internal_construct() returns an Object, and not Value! This has been
a source of confusion for a long time, since in the spec there's
always an Object returned but the Value return type in LibJS meant
that this could not be fully trusted and something could screw you
over.
- Arguments are passed explicitly in form of a MarkedValueList,
allowing manipulation (BoundFunction). We still put them on the
execution context as a lot of code depends on it (VM::arguments()),
but not from the Call() / Construct() AOs anymore, which now allows
for bypassing them and invoking [[Call]] / [[Construct]] directly.
Nothing but Call() / Construct() themselves do that at the moment,
but future additions to ECMA262 or already existing web specs might.
- Spec compliant, standalone Call() and Construct() AOs: currently the
closest we have is VM::{call,construct}(), but those try to cater to
all the different function object subclasses at once, resulting in a
horrible mess and calling AOs with functions they should never be
called with; most prominently PrepareForOrdinaryCall and
OrdinaryCallBindThis, which are only for ECMAScriptFunctionObject.
As a result this also contains an implicit optimization: we no longer
need to create a new function environment for NativeFunctions - which,
worth mentioning, is what started this whole crusade in the first place
:^)
Since non-strict direct eval() can insert new bindings into a
surrounding var scope, we cannot safely cache some assumptions about
environment chain layout after eval() has taken place.
Since eval() is rare, let's do what other engines do and simply
deoptimize in its presence. This patch adds a new "permanently screwed"
flag to JS::Environment that will be set on the entire variable
environment chain upon non-strict direct eval().
This gives FunctionNode a "might need arguments object" boolean flag and
sets it based on the simplest possible heuristic for this: if we
encounter an identifier called "arguments" or "eval" up to the next
(nested) function declaration or expression, we won't need an arguments
object. Otherwise, we *might* need one - the final decision is made in
the FunctionDeclarationInstantiation AO.
Now, this is obviously not perfect. Even if you avoid eval, something
like `foo.arguments` will still trigger a false positive - but it's a
start and already massively cuts down on needlessly allocated objects,
especially in real-world code that is often minified, and so a full
"arguments" identifier will be an actual arguments object more often
than not.
To illustrate the actual impact of this change, here's the number of
allocated arguments objects during a full test-js run:
Before:
- Unmapped arguments objects: 78765
- Mapped arguments objects: 2455
After:
- Unmapped arguments objects: 18
- Mapped arguments objects: 37
This results in a ~5% speedup of test-js on my Linux host machine, and
about 3.5% on i686 Serenity in QEMU (warm runs, average of 5).
The following microbenchmark (calling an empty function 1M times) runs
25% faster on Linux and 45% on Serenity:
function foo() {}
for (var i = 0; i < 1_000_000; ++i)
foo();
test262 reports no changes in either direction, apart from a speedup :^)
Before this we used an ad-hoc combination of references and 'variables'
stored in a hashmap. This worked in most cases but is not spec like.
Additionally hoisting, dynamically naming functions and scope analysis
was not done properly.
This patch fixes all of that by:
- Implement BindingInitialization for destructuring assignment.
- Implementing a new ScopePusher which tracks the lexical and var
scoped declarations. This hoists functions to the top level if no
lexical declaration name overlaps. Furthermore we do checking of
redeclarations in the ScopePusher now requiring less checks all over
the place.
- Add methods for parsing the directives and statement lists instead
of having that code duplicated in multiple places. This allows
declarations to pushed to the appropriate scope more easily.
- Remove the non spec way of storing 'variables' in
DeclarativeEnvironment and make Reference follow the spec instead of
checking both the bindings and 'variables'.
- Remove all scoping related things from the Interpreter. And instead
use environments as specified by the spec. This also includes fixing
that NativeFunctions did not produce a valid FunctionEnvironment
which could cause issues with callbacks and eval. All
FunctionObjects now have a valid NewFunctionEnvironment
implementation.
- Remove execute_statements from Interpreter and instead use
ASTNode::execute everywhere this simplifies AST.cpp as you no longer
need to worry about which method to call.
- Make ScopeNodes setup their own environment. This uses four
different methods specified by the spec
{Block, Function, Eval, Global}DeclarationInstantiation with the
annexB extensions.
- Implement and use NamedEvaluation where specified.
Additionally there are fixes to things exposed by these changes to eval,
{for, for-in, for-of} loops and assignment.
Finally it also fixes some tests in test-js which where passing before
but not now that we have correct behavior :^).
Clang was failing because because it rightfully saw we were attempting
to call a deleted constructor of `MarkedValueList`. If you explicitly
called move(list) then GCC would complain that the move was unnecessary.
For what ever reason both tool chains accept when we construct the
ThrowCompletionOr explicitly that we move the list into and return that.
This is much more common across the whole codebase and even these two
files. The same is used to return an empty JS::Value in an exception
check, for example.
This is where the spec wants to have it. Requires a couple of hacks as
currently everything that needs a Realm actually has a GlobalObject, so
we need to go via the Interpreter.
The primary themes here are invoking js_string() with existing instances
of Utf16String when possible, and not creating entire UTF-8 copies when
not needed.
This commit does not go out of its way to reduce copying of the string
data yet, but is a minimum set of changes to compile LibJS after making
PrimitiveString hold a Utf16String.
This also converts the GetSubstitution abstract operation take its input
strings as UTF-16 now that all callers are UTF-16 capable. This means
String.prototype.replace (and replaceAll) no longer needs UTF-8 and
UTF-16 copies of these strings.
- Fix evaluation order: IsArray(O) should always be called and before
Get(O, @@toStringTag), previously it was the other way around and
IsArray would only be called if @@toStringTag is not a string
- Add missing exception checks to both function calls
- Add missing builtin tag for arguments object
Also, while we're here:
- Update variable names to match spec
- Add spec step comments
Now that the Object rewrite is in place, we have enough tools to
implement the mapped `arguments` propreties according to spec.
The basic mechanism is that the `arguments` object installs a hidden
parameter mapping object that property accesses get filtered through.
This is how accessing numeric properties on `arguments` are proxied
to the named identifier in the function scope.
When `arguments` is instantiated, getters and setters are created
for all the numeric properties on the object that correspond to
function arguments. These getters and setters can be deleted from the
object. This is all pretty intricate, so refer to the spec for details.
Note that the `arguments` object itself is still lazily instantiated
on first access within a function. This is non-conforming, and we'll
have to revisit this once we get around to improving function calls.
This is a huge patch, I know. In hindsight this perhaps could've been
done slightly more incremental, but I started and then fixed everything
until it worked, and here we are. I tried splitting of some completely
unrelated changes into separate commits, however. Anyway.
This is a rewrite of most of Object, and by extension large parts of
Array, Proxy, Reflect, String, TypedArray, and some other things.
What we already had worked fine for about 90% of things, but getting the
last 10% right proved to be increasingly difficult with the current code
that sort of grew organically and is only very loosely based on the
spec - this became especially obvious when we started fixing a large
number of test262 failures.
Key changes include:
- 1:1 matching function names and parameters of all object-related
functions, to avoid ambiguity. Previously we had things like put(),
which the spec doesn't have - as a result it wasn't always clear which
need to be used.
- Better separation between object abstract operations and internal
methods - the former are always the same, the latter can be overridden
(and are therefore virtual). The internal methods (i.e. [[Foo]] in the
spec) are now prefixed with 'internal_' for clarity - again, it was
previously not always clear which AO a certain method represents,
get() could've been both Get and [[Get]] (I don't know which one it
was closer to right now).
Note that some of the old names have been kept until all code relying
on them is updated, but they are now simple wrappers around the
closest matching standard abstract operation.
- Simplifications of the storage layer: functions that write values to
storage are now prefixed with 'storage_' to make their purpose clear,
and as they are not part of the spec they should not contain any steps
specified by it. Much functionality is now covered by the layers above
it and was removed (e.g. handling of accessors, attribute checks).
- PropertyAttributes has been greatly simplified, and is being replaced
by PropertyDescriptor - a concept similar to the current
implementation, but more aligned with the actual spec. See the commit
message of the previous commit where it was introduced for details.
- As a bonus, and since I had to look at the spec a whole lot anyway, I
introduced more inline comments with the exact steps from the spec -
this makes it super easy to verify correctness.
- East-const all the things.
As a result of all of this, things are much more correct but a bit
slower now. Retaining speed wasn't a consideration at all, I have done
no profiling of the new code - there might be low hanging fruits, which
we can then harvest separately.
Special thanks to Idan for helping me with this by tracking down bugs,
updating everything outside of LibJS to work with these changes (LibWeb,
Spreadsheet, HackStudio), as well as providing countless patches to fix
regressions I introduced - there still are very few (we got it down to
5), but we also get many new passing test262 tests in return. :^)
Co-authored-by: Idan Horowitz <idan.horowitz@gmail.com>
This patch implements spec-compliant runtime semantics for the following
constructs:
- super.property
- super[property]
The MakeSuperPropertyReference AO is added to support this. :^)
This patch adds a new ArgumentsObject class to represent what the spec
calls "Arguments Exotic Objects"
These are constructed by the new CreateMappedArgumentsObject when the
`arguments` identifier is resolved in a callee context.
The implementation is incomplete and doesn't yet support mapping of
the parameter variables to the indexed properties of `arguments`.
This was a standalone function previously (get_method()), but instead of
passing a Value to it, we can just make it a method.
Also add spec step comments and fix the receiver value by using GetV().
Requires a bunch of find-and-replace updates across LibJS, but
constructing a PropertyName from a nullptr Symbol* should not be
possible - let's enforce this at the compiler level instead of using
VERIFY() (and already dereference Symbol pointers at the call site).
This is true for environments created by `with` statements, and false
for other (global) object environments.
Also add the WithBaseObject abstract operation while we're here.
To better follow the spec, we need to distinguish between the current
execution context's lexical environment and variable environment.
This patch moves us to having two record pointers, although both of
them point at the same environment records for now.
This patch adds FunctionEnvironmentRecord as a subclass of the existing
DeclarativeEnvironmentRecord. Things that are specific to function
environment records move into there, simplifying the base.
Most of the abstract operations related to function environment records
are rewritten to match the spec exactly. I also had to implement
GetThisEnvironment() and GetSuperConstructor() to keep tests working
after the changes, so that's nice as well. :^)
Value.{cpp,h} has become a dumping ground, let's change that.
Things that are directly related to Values (e.g. bitwise/binary ops,
equality related functions) can remain, but everything else that's not a
Value or Object method and globally required (not just a static function
somewhere) is being moved.
Also convert to east-const while we're here.
I haven't touched IteratorOperations.{cpp,h}, it seems fine to still
have those separately.
