Store parser errors, source range filenames, source code filenames,
module source, and Rust parser errors as UTF-16 where they flow back
into JavaScript-visible strings. Keep byte-oriented source buffers
byte-backed.
Remove temporary PrimitiveString, ByteString, and UTF-8 detours from
JSON, RegExp, module debug logging, print formatting, and tests.
Produce JS-visible string results as UTF-16 at their source, including
numeric formatting, BigInt and BigFraction formatting, URI encoding,
console formatting, parser errors, regular expression errors, Intl and
Temporal records, LibUnicode locale boundaries, and LibWeb bindings.
Handle fractional radix formatting through the UTF-16 builder view.
Thread UTF-16 string input through JSON, script parsing, Date parsing,
Intl option parsing, Temporal parsing, and the helper library boundaries
that feed those parsers. Preserve ASCII fast paths where the source data
is known to be ASCII.
Use the Rust bytecode dumper's basic block collection logic for the
metadata block count. This removes the last C++ bytecode label walk and
lets us delete the generated C++ label and operand visitor helpers.
Generate Rust bytecode dump helpers from Bytecode.def and route
Executable::dump() through them for instruction stream formatting.
Add a small Rust runtime::value helper for decoding encoded LibJS
Values so immediate Value operands are formatted on the Rust side. C++
callbacks remain only for local names and GC-backed Value payloads that
still need LibJS object access.
Remove the generated C++ to_byte_string_impl() methods and the old
Instruction::to_byte_string() dispatch. The bytecode dump tests cover
output compatibility.
Add a ref-counted decoded bytecode cache backing so bytecode cache
materialization can create fresh script or module records from a shared
decoded sidecar without passing around one-shot raw blob ownership.
Keep that backing in ExecutableBacking for records materialized from
bytecode cache sidecars, so the immutable decoded data stays alive for
as long as the installed record needs it.
Cover the shared backing path with a bytecode-cache test that
materializes and runs two scripts from one decoded backing.
Warm cache hits used to validate bytecode cache blobs on the main
thread. Route script and module sidecars through a worker step that
decodes and validates the cache blob, then returns the validated blob
for main-thread materialization.
Keep the source bytes mmap-backed and avoid decoding the full source on
cache hits. The main thread still computes the UTF-16 source length so
validation can reject stale blobs before materialization.
Remove the decoded source length getter now that sidecar validation uses
the explicit validation API instead.
Add an explicit validation entry point for decoded bytecode cache blobs.
This lets callers validate a blob before materialization while keeping
the existing validated-before-use invariant in place.
Make validation idempotent so a prevalidated blob is not walked again
when materialization reaches the same check. Keep the existing decoded
source length query for callers that still validate synchronously.
Route decoded cache blob owner destruction through the origin event
loop. This lets a worker decode or reject mapped bytecode without
releasing non-atomically refcounted ImmutableBytes off-thread, while
lazy cached function materialization can still retain mmap-backed
bytecode.
The Rust pipeline is always available now, so the getter only wrapped
code that always ran. Remove it and make the JavaScript fetch paths use
the off-thread Rust pipeline directly.
This also removes the unreachable synchronous fallback branches from the
classic script and module fetch paths.
Switch bytecode cache source identity from decoded UTF-16 source text to
the original encoded response bytes plus the effective source encoding.
Store the decoded source length in the cache blob header so warm loads
can build lazy SourceCode objects without decoding the source before
checking the sidecar.
This removes the main-thread decoded_source_text_info pass from valid
warm-cache script and module loads. The source is only decoded on cache
miss, or when a rejected sidecar falls back to source compilation.
Give scripts and modules an explicit list of their shared function data
records, and teach Rust materialization to populate that list directly.
This gives bytecode cache installation a stable set of functions to
update when replacing executable backing storage.
Keep executable bytecode payloads decoded from owner-backed bytecode
cache blobs as ranges into the original blob instead of copying them
into Rust Vec allocations. The mapped blob owner is held by decoded
executable records, including lazy nested function executables, so the
borrowed bytecode remains alive until materialization copies it into the
final C++ Executable.
Use the owner-backed decoder for HTTP bytecode cache hits and keep the
plain byte decoder for tests and in-memory callers. Add coverage for
materializing bytecode cache data from an ImmutableBytes mapped file.
Keep fully compiled function bytecode in its Rust-side form until the
function is called for the first time. This covers decoded disk cache
records and freshly precompiled bytecode, so startup avoids eagerly
allocating every nested function executable.
Validate cached function bytecode before accepting a cache entry. This
keeps the existing failure behavior for corrupt on-disk cache data. Add
coverage for bytecode-cache and freshly precompiled functions to assert
that nested executables stay absent after script materialization, then
appear after the function is called.
Create parser-free script and module materializers for decoded cache
blobs. Cached functions create SFDs without Rust compile inputs and
attach their precompiled executable immediately, while declaration
metadata is populated from decoded records.
Treat cache blobs as external input from the HTTP disk cache. Run
bytecode validation unconditionally before fixing up cache pointers, and
reject decoded source ranges or metadata indices that would be
out-of-bounds during C++ materialization.
Report executable validation failures as parser errors so callers can
reject corrupt sidecars and fall back to source compilation. LibJS tests
cover corrupt top-level bytecode, declaration bytecode, and declaration
source spans.
Store a SHA-256 fingerprint of the decoded source text in each bytecode
cache blob, and require callers to provide the expected fingerprint when
validating or decoding a blob.
This rejects sidecars for stale HTTP cache entries whose URL and request
headers still match but whose source body has been replaced. Bytecode
cache tests cover the mismatched-source rejection path.
Expose an owned decoded bytecode cache handle through RustIntegration.
This lets C++ callers keep validated metadata and executable records in
Rust-owned cache structures without invoking the parser.
Extend the js bytecode cache validation mode to create and free the FFI
handle so blob generation exercises the ownership path.
Record whether each bytecode cache blob contains a classic script or a
module, and pass that type through the serializer call sites.
Require validation callers to provide the expected program type so
script and module sidecars cannot be reused for the wrong loader.
Add a versioned Rust bytecode cache writer for fully compiled programs.
The blob records executable bytecode, metadata tables, source maps,
exception handlers, nested function bytecode, and class blueprints
without materializing GC objects.
Expose the serialized blob through RustIntegration as an owned
ByteBuffer so Web-facing callers can store it as HTTP cache data.
Keep fetched script and module compilation on the latency-sensitive path
limited to top-level code and eager direct IIFE compilation before
returning bytecode to the main thread.
Add a separate full off-thread compile entry point for bytecode cache
generation. Cache jobs can use it to compile every nested function after
the execution path has already been unblocked.
Add Rust and C++ integration points for cloning lazy function compile
payloads, compiling them to GC-free bytecode off the main thread, and
materializing the result later on the main thread.
The cloned payload lets background compilation race with lazy
main-thread compilation without sharing AST ownership between threads.
Compiled function artifacts recursively include nested functions, so
materialization can discard the corresponding AST subtree.
Split Rust program compilation so code generation and assembly finish
before the main thread materializes GC-backed executable objects. The
new CompiledProgram handle owns the parsed program, generator state, and
bytecode until C++ consumes it on the main thread.
Wire WebContent script fetching through that handle for classic scripts
and modules. Syntax-error paths still return ParsedProgram, so existing
error reporting stays in place. Successful fetches now do top-level
codegen on the thread pool before deferred_invoke hands control back to
the main thread.
Executable creation, SharedFunctionInstanceData materialization, module
metadata extraction, and declaration data extraction still run on the
main thread where VM and GC access is valid.
The proposal has not seemed to progress for a while, and there is
a open issue about module imports which breaks HTML integration.
While we could probably make an AD-HOC change to fix that issue,
it is deep enough in the JS engine that I am not particularly
keen on making that change.
Until other browsers begin to make positive signals about shipping
ShadowRealms, let's remove our implementation for now.
There is still some cleanup that can be done with regard to the
HTML integration, but there are a few more items that need to be
untangled there.
Delete AST.cpp, AST.h, ASTDump.cpp, ScopeRecord.h, and the dead
get_builtin(MemberExpression const&) from Builtins.cpp.
Extract ImportEntry and ExportEntry into a new ModuleEntry.h,
since they are data types used by the module system, not AST
node types.
Inline ModuleRequest's sorting constructor and
SourceRange::filename().
Remove the dead annex_b_function_declarations field from
EvalDeclarationData, which was only populated by the C++ parser.
Delete Parser.cpp/h and ScopeCollector.cpp/h, now that all parsing
goes through the Rust pipeline.
Port test262-runner to use RustIntegration::parse_program() for its
fast parse-only check instead of the C++ Parser.
Add parsed_program_has_errors() and free_parsed_program() to the
RustIntegration public API for parse-only use cases.
Port all remaining users of the C++ Parser/Lexer/Generator to
use the Rust pipeline instead:
- Intrinsics: Remove C++ fallback in parse_builtin_file()
- ECMAScriptFunctionObject: Remove C++ compile() fallback
- NativeJavaScriptBackedFunction: Remove C++ compile() fallback
- EventTarget: Port to compile_dynamic_function
- WebDriver/ExecuteScript: Port to compile_dynamic_function
- LibTest/JavaScriptTestRunner.h: Remove Parser/Lexer includes
- FuzzilliJs: Remove unused Parser/Lexer includes
Also remove the dead Statement-based template instantiation of
async_block_start/async_function_start.
Remove PipelineComparison.cpp/h and all LIBJS_COMPARE_PIPELINES
support from RustIntegration.cpp. This includes:
- The compare_pipelines_enabled() function
- All comparison blocks in compile_script/eval/module/function
- The pair_shared_function_data() helper
- The m_cpp_comparison_sfd field on SharedFunctionInstanceData
The Rust pipeline has been validated extensively through comparison
testing and no longer needs the side-by-side verification harness.
Replace the BytecodeFactory header with cbindgen.
This will help ensure that types and enums and constants are kept in
sync between the C++ and Rust code. It's also a step in exporting more
Rust enums directly rather than relying on magic constants for
switch statements.
The FFI functions are now all placed in the JS::FFI namespace, which
is the cause for all the churn in the scripting parts of LibJS and
LibWeb.
Add compile_parsed_module() to RustIntegration, which takes a
RustParsedProgram and a SourceCode (from parse_program with
ProgramType::Module) and compiles it on the main thread with GC
interaction.
Rewrite compile_module() to use the new split functions internally.
Add SourceTextModule::parse_from_pre_parsed() and
JavaScriptModuleScript::create_from_pre_parsed() to allow creating
module scripts from a pre-parsed RustParsedProgram.
This prepares the infrastructure for off-thread module parsing.
Create a SourceCode on the main thread (performing UTF-8 to UTF-16
conversion), then submit parse_program() to the ThreadPool for
Rust parsing on a worker thread. This unblocks the WebContent event
loop during external script loading.
Add Script::create_from_parsed() and
ClassicScript::create_from_pre_parsed() factory methods that take a
pre-parsed RustParsedProgram and a SourceCode, performing only the
GC-allocating compile step on the main thread.
Falls back to synchronous parsing when the Rust pipeline is
unavailable (LIBJS_CPP=1 or LIBJS_COMPARE_PIPELINES=1).
Expose the Rust parse/compile split to C++ callers:
- parse_program(): takes raw UTF-16 data and a ProgramType
parameter (Script or Module). No GC interaction, thread-safe.
- compile_parsed_script(): takes a pre-parsed RustParsedProgram
and a SourceCode, checks for errors, and calls
rust_compile_parsed_script(). Returns a ScriptResult.
Rewrite compile_script() to use the split path internally. The
pipeline comparison logic now gets the AST dump from the
ParsedProgram before compilation consumes it.
Implement a complete Rust reimplementation of the LibJS frontend:
lexer, parser, AST, scope collector, and bytecode code generator.
The Rust pipeline is built via Corrosion (CMake-Cargo bridge) and
linked into LibJS as a static library. It is gated behind a build
flag (ENABLE_RUST, on by default except on Windows) and two runtime
environment variables:
- LIBJS_CPP: Use the C++ pipeline instead of Rust
- LIBJS_COMPARE_PIPELINES=1: Run both pipelines in lockstep,
aborting on any difference in AST or bytecode generated.
The C++ side communicates with Rust through a C FFI layer
(RustIntegration.cpp/h) that passes source text to Rust and receives
a populated Executable back via a BytecodeFactory interface.
