Rather than printing them to stderr directly the parser now keeps a
Vector<Error>, which allows the "owner" of the parser to consume them
individually after parsing.
The Error struct has a message, line number, column number and a
to_string() helper function to format this information into a meaningful
error message.
The Function() constructor will now include an error message when
throwing a SyntaxError.
There are many cases which shouldn't even parse, like
null = ...
true = ...
false = ...
123 = ...
"foo" = ...
However this *is* valid syntax:
foo() = ...
So we still have to keep the current code doing a runtime check if the
LHS value is a resolvable reference. I believe this was declared valid
syntax to *in theory* allow functions returning references - though in
practice that isn't a thing.
Fixes#2204.
This required 2 changes:
1. In the parser, create a new variable scope, so the variable is
declared in it instead of the scope in which the 'for' is found.
2. On execute, push the variable into the newly created block. Existing
code created an empty block (no variables, no arguments) which
allows Interpreter::enter_scope() to skip the creation of a new
environment, therefore when the variable initializer is executed, it
sets the variable to the outer scope. By attaching the variable to
the new block, the block gets a new environment.
This is only needed for 'let' / 'const' declarations, since 'var'
declarations are expected to leak.
Fixes: #2103
"[Function.length is] the number of formal parameters. This number
excludes the rest parameter and only includes parameters before
the first one with a default value." - MDN
To make processing tagged template literals easier, template literals
will now add one empty StringLiteral before and after each template
expression *if* there's no other string - e.g.:
`${foo}` -> "", foo, ""
`test${foo}${bar}test` -> "test", foo, "", bar, "test"
This also matches the behaviour of many other parsers.
A regression was introduced in dc9b4da where the parser would
incorrectly parse the assignment of arrow functions to (non-declaration)
variables. For example, consider:
a = () => {}
Because the parser was aware of default parameters, in
try_parse_arrow_function, the equals sign would be interpreted as a
default argument, leading to incorrect parsing of the overall
expression. Also resulted in some funny behavior
(a = () => {} => {} worked just fine!).
The simple fix is to only look for default parameters if the arrow
function is required to have parenthesis.
Adds fully functioning template literals. Because template literals
contain expressions, most of the work has to be done in the Lexer rather
than the Parser. And because of the complexity of template literals
(expressions, nesting, escapes, etc), the Lexer needs to have some
template-related state.
When entering a new template literal, a TemplateLiteralStart token is
emitted. When inside a literal, all text will be parsed up until a '${'
or '`' (or EOF, but that's a syntax error) is seen, and then a
TemplateLiteralExprStart token is emitted. At this point, the Lexer
proceeds as normal, however it keeps track of the number of opening
and closing curly braces it has seen in order to determine the close
of the expression. Once it finds a matching curly brace for the '${',
a TemplateLiteralExprEnd token is emitted and the state is updated
accordingly.
When the Lexer is inside of a template literal, but not an expression,
and sees a '`', this must be the closing grave: a TemplateLiteralEnd
token is emitted.
The state required to correctly parse template strings consists of a
vector (for nesting) of two pieces of information: whether or not we
are in a template expression (as opposed to a template string); and
the count of the number of unmatched open curly braces we have seen
(only applicable if the Lexer is currently in a template expression).
TODO: Add support for template literal newlines in the JS REPL (this will
cause a syntax error currently):
> `foo
> bar`
'foo
bar'
We already skipped random semicolons in Parser::parse_program(), but now
they are properly matched and parsed as empty statements - and thus
recognized as a valid body of an if / else / while / ... statement.
Adds the ability for function arguments to have default values. This
works for standard functions as well as arrow functions. Default values
are not printed in a <function>.toString() call, as nodes cannot print
their source string representation.
Instead of having fprintf()s all over the place we can now use
syntax_error("message") or syntax_error("message", line, column).
This takes care of a consistent format, appending a newline and getting
the line number and column of the current token if the last two params
are omitted.
Implement the syntax and behavor necessary to support array literals
such as [...[1, 2, 3]]. A type error is thrown if the target of the
spread operator does not evaluate to an array (though it should
eventually just check for an iterable).
Note that the spread token's name is TripleDot, since the '...' token is
used for two features: spread and rest. Calling it anything involving
'spread' or 'rest' would be a bit confusing.
It turns out "delete" is actually a unary op :)
This patch implements deletion of object properties, it doesn't yet
work for casually deleting properties from the global object.
When deleting a property from an object, we switch that object to
having a unique shape, no longer sharing shapes with others.
Once an object has a unique shape, it no longer needs to care about
shape transitions.
This is required for template literals - we're not quite there yet, but at
least the parser can now tell us when this token is encountered -
currently this yields "Unexpected token Invalid". Not really helpful.
The character is a "backtick", but as we already have
TokenType::{StringLiteral,RegexLiteral} this seemed like a fitting name.
This also enables syntax highlighting for template literals in the js
REPL and LibGUI's JSSyntaxHighlighter.
"var" declarations are hoisted to the nearest function scope, while
"let" and "const" are hoisted to the nearest block scope.
This is done by the parser, which keeps two scope stacks, one stack
for the current var scope and one for the current let/const scope.
When the interpreter enters a scope, we walk all of the declarations
and insert them into the variable environment.
We don't support the temporal dead zone for let/const yet.
Many other parsers call it with this name.
Also Type can be confusing in this context since the DeclarationType is
not the type (number, string, etc.) of the variables that are being
declared by the VariableDeclaration.
While debugging test failures, it's pretty frustrating to have to go do
printf debugging to figure out what test is failing right now. While
watching your JS Raytracer stream it seemed like this was pretty
furstrating as well. So I wanted to start working on improving the
diagnostics here.
In the future I hope we can eventually be able to plumb the info down
to the Error classes so any thrown exceptions will contain enough
metadata to know where they came from.
This patch adds support in the parser and interpreter for this:
var a = 1, b = 2, c = a + b;
VariableDeclaration is now a sequence of VariableDeclarators. :^)
The Lexer and AST already have all the functionality required in place,
so this is just updating Parser::match_secondary_expression() and
Parser::parse_expression() to handle TokenType::{Ampersand,Pipe,Caret},
as well as adding some tests.
There is no such thing as a "undefined literal" in JS - undefined is
just a property on the global object with a value of undefined.
This is pretty similar to NaN.
var undefined = "foo"; is a perfectly fine AssignmentExpression :^)
These functions allow us to try to parse ambiguous expressions (such as
arrow function arguments in parentheses), and
rewind the state of the Parser if an expression candidate failed to
parse.
This operator walks the prototype chain of the RHS value and looks for
a "prototype" property with the same value as the prototype of the LHS.
This is pretty cool. :^)
NewExpression mostly piggybacks on the existing CallExpression. The big
difference is that "new" creates a new Object and passes it as |this|
to the callee.
Unary expressions parsing now respects precedence and associativity of
operators. This patch also makes `typeof` left-associative which was
an oversight.
Thanks to Conrad for helping me work this out. :^)
You can now throw an expression to the nearest catcher! :^)
To support throwing arbitrary values, I added an Exception class that
sits as a wrapper around whatever is thrown. In the future it will be
a logical place to store a call stack.
We can now handle scripts with if/else in LibJS. Most of the changes
are about fixing IfStatement to store the consequent and alternate node
as Statements.
Interpreter now also runs Statements, rather than running ScopeNodes.
FunctionExpression is mostly like FunctionDeclaration, except the name
is optional. Share the parsing logic in parse_function_node<NodeType>.
This allows us to do nice things like:
document.addEventListener("DOMContentLoaded", function() {
alert("Hello friends!");
});
This also tightens the means of redeclaration of a variable by proxy,
since we now have a way of knowing how a variable was initially
declared, we can check if it was declared using `let` or `const` and
not tolerate redeclaration like we did previously.
Remove the need to construct a full Value during parsing. This means
we don't have to worry about plumbing the heap into the parser.
The Literal ASTNode now has a bunch of subclasses that synthesize a
Value on demand.
This adds a basic Javascript lexer and parser. It can parse the
currently existing demo programs. More work needs to be done to
turn it into a complete parser than can parse arbitrary JS Code.
The lexer outputs tokens with preceeding whitespace and comments
in the trivia member. This should allow us to generate the exact
source code by concatenating the generated tokens.
The parser is written in a way that it always returns a complete
syntax tree. Error conditions are represented as nodes in the
tree. This simplifies the code and allows it to be used as an
early stage parser, e.g for parsing JS documents in an IDE while
editing the source code.:
