dmaax/ladybird
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ladybird/Libraries/LibJS/SourceTextModule.h

111 lines
6 KiB
C
Raw Permalink Normal View History

LibJS: Add a barebones SourceTextModule class This corresponds to "Source Text Module Record" from the spec.
2021-09-14 15:51:16 -03:00
/*
Meta: Update my e-mail address everywhere
2024-10-04 08:19:50 -03:00
* Copyright (c) 2021-2023, Andreas Kling <andreas@ladybird.org>
LibJS: Implement Module linking and evaluating
2022-01-18 15:29:17 -03:00
* Copyright (c) 2022, David Tuin <davidot@serenityos.org>
LibJS: Add a barebones SourceTextModule class This corresponds to "Source Text Module Record" from the spec.
2021-09-14 15:51:16 -03:00
*
* SPDX-License-Identifier: BSD-2-Clause
*/
#pragma once
LibWeb+LibJS: Cache decoded JS bytecode sidecars Add a ref-counted decoded bytecode cache backing so bytecode cache materialization can create fresh script or module records from a shared decoded sidecar without passing around one-shot raw blob ownership. Keep that backing in ExecutableBacking for records materialized from bytecode cache sidecars, so the immutable decoded data stays alive for as long as the installed record needs it. Cover the shared backing path with a bytecode-cache test that materializes and runs two scripts from one decoded backing.
2026-05-28 14:09:12 -03:00
#include <AK/NonnullRefPtr.h>
LibJS: Implement Module linking and evaluating
2022-01-18 15:29:17 -03:00
#include <LibJS/CyclicModule.h>
LibJS: Route bytecode cache installs through owners Store ExecutableBacking on Script and SourceTextModule, and make those owners coordinate generated cache installation. Replacing executable backing now clears non-cache compile inputs and verifies that mapped cache-backed records no longer retain source or heap bytecode inputs.
2026-05-21 16:50:08 -03:00
#include <LibJS/ExecutableBacking.h>
LibJS: Enable EXPLICIT_SYMBOL_EXPORT and annotate minimum symbol set
2025-07-19 17:49:30 -03:00
#include <LibJS/Export.h>
LibJS: Add a barebones SourceTextModule class This corresponds to "Source Text Module Record" from the spec.
2021-09-14 15:51:16 -03:00
#include <LibJS/Forward.h>
LibJS: Remove C++ AST Delete AST.cpp, AST.h, ASTDump.cpp, ScopeRecord.h, and the dead get_builtin(MemberExpression const&) from Builtins.cpp. Extract ImportEntry and ExportEntry into a new ModuleEntry.h, since they are data types used by the module system, not AST node types. Inline ModuleRequest's sorting constructor and SourceRange::filename(). Remove the dead annex_b_function_declarations field from EvalDeclarationData, which was only populated by the C++ parser.
2026-03-19 15:37:46 -03:00
#include <LibJS/ModuleEntry.h>
LibJS: Make CyclicModule.h not include PromiseCapability.h
2022-11-23 07:41:19 -03:00
#include <LibJS/Runtime/ExecutionContext.h>
LibJS: Add a barebones SourceTextModule class This corresponds to "Source Text Module Record" from the spec.
2021-09-14 15:51:16 -03:00
namespace JS {
LibJS: Generate FFI header using cbindgen instead of hand-rolling Replace the BytecodeFactory header with cbindgen. This will help ensure that types and enums and constants are kept in sync between the C++ and Rust code. It's also a step in exporting more Rust enums directly rather than relying on magic constants for switch statements. The FFI functions are now all placed in the JS::FFI namespace, which is the cause for all the churn in the scripting parts of LibJS and LibWeb.
2026-03-17 18:47:58 -03:00
namespace FFI {
struct ParsedProgram;
LibWeb+LibJS: Compile fetched top-level JS off-thread Split Rust program compilation so code generation and assembly finish before the main thread materializes GC-backed executable objects. The new CompiledProgram handle owns the parsed program, generator state, and bytecode until C++ consumes it on the main thread. Wire WebContent script fetching through that handle for classic scripts and modules. Syntax-error paths still return ParsedProgram, so existing error reporting stays in place. Successful fetches now do top-level codegen on the thread pool before deferred_invoke hands control back to the main thread. Executable creation, SharedFunctionInstanceData materialization, module metadata extraction, and declaration data extraction still run on the main thread where VM and GC access is valid.
2026-04-26 09:53:34 -03:00
struct CompiledProgram;
LibWeb+RequestServer: Send cached bytecode with responses Attach cached JavaScript bytecode sidecars to HTTP response headers so WebContent can materialize classic and module scripts directly from a decoded cache blob on cache hits. Carry the disk cache vary key with the sidecar and reuse it when storing fresh bytecode, avoiding mismatches against the augmented network request headers used to create the cache entry. Keep CORS-filtered module responses intact for status, MIME, and script creation checks. Read bytecode sidecar data only from the internal response, and treat decode or materialization failure as a cache miss that falls back to normal source compilation.
2026-05-03 15:04:35 -03:00
struct DecodedBytecodeCacheBlob;
LibJS: Generate FFI header using cbindgen instead of hand-rolling Replace the BytecodeFactory header with cbindgen. This will help ensure that types and enums and constants are kept in sync between the C++ and Rust code. It's also a step in exporting more Rust enums directly rather than relying on magic constants for switch statements. The FFI functions are now all placed in the JS::FFI namespace, which is the cause for all the churn in the scripting parts of LibJS and LibWeb.
2026-03-17 18:47:58 -03:00
}
LibWeb+LibJS: Cache decoded JS bytecode sidecars Add a ref-counted decoded bytecode cache backing so bytecode cache materialization can create fresh script or module records from a shared decoded sidecar without passing around one-shot raw blob ownership. Keep that backing in ExecutableBacking for records materialized from bytecode cache sidecars, so the immutable decoded data stays alive for as long as the installed record needs it. Cover the shared backing path with a bytecode-cache test that materializes and runs two scripts from one decoded backing.
2026-05-28 14:09:12 -03:00
namespace RustIntegration {
class DecodedBytecodeCache;
Libraries: Clean up UTF-16 source text paths Store parser errors, source range filenames, source code filenames, module source, and Rust parser errors as UTF-16 where they flow back into JavaScript-visible strings. Keep byte-oriented source buffers byte-backed. Remove temporary PrimitiveString, ByteString, and UTF-8 detours from JSON, RegExp, module debug logging, print formatting, and tests.
2026-06-21 14:03:19 -03:00
struct ModuleResult;
LibWeb+LibJS: Cache decoded JS bytecode sidecars Add a ref-counted decoded bytecode cache backing so bytecode cache materialization can create fresh script or module records from a shared decoded sidecar without passing around one-shot raw blob ownership. Keep that backing in ExecutableBacking for records materialized from bytecode cache sidecars, so the immutable decoded data stays alive for as long as the installed record needs it. Cover the shared backing path with a bytecode-cache test that materializes and runs two scripts from one decoded backing.
2026-05-28 14:09:12 -03:00
}
LibJS: Add a barebones SourceTextModule class This corresponds to "Source Text Module Record" from the spec.
2021-09-14 15:51:16 -03:00
// 16.2.1.6 Source Text Module Records, https://tc39.es/ecma262/#sec-source-text-module-records
LibJS: Enable EXPLICIT_SYMBOL_EXPORT and annotate minimum symbol set
2025-07-19 17:49:30 -03:00
class JS_API SourceTextModule final : public CyclicModule {
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
GC_CELL(SourceTextModule, CyclicModule);
GC_DECLARE_ALLOCATOR(SourceTextModule);
LibJS: Make Script and Module GC-allocated This ensures that code currently in any active or saved execution stack always stays alive.
2022-09-05 09:31:25 -03:00
LibJS: Add a barebones SourceTextModule class This corresponds to "Source Text Module Record" from the spec.
2021-09-14 15:51:16 -03:00
public:
Everywhere: Write dtors for types with incomplete members out-of-line These are rejected by Clang (19) trunk as a result of https://github.com/llvm/llvm-project/pull/77753.
2024-06-16 05:38:35 -03:00
virtual ~SourceTextModule() override;
Libraries: Clean up UTF-16 source text paths Store parser errors, source range filenames, source code filenames, module source, and Rust parser errors as UTF-16 where they flow back into JavaScript-visible strings. Keep byte-oriented source buffers byte-backed. Remove temporary PrimitiveString, ByteString, and UTF-8 detours from JSON, RegExp, module debug logging, print formatting, and tests.
2026-06-21 14:03:19 -03:00
static Result<GC::Ref<SourceTextModule>, Vector<ParserError>> parse(Utf16View source_text, Realm&, StringView filename = {}, Utf16View display_filename = {}, Script::HostDefined* host_defined = nullptr);
static Result<GC::Ref<SourceTextModule>, Vector<ParserError>> parse(NonnullRefPtr<SourceCode const>, Realm&, StringView filename = {}, Script::HostDefined* host_defined = nullptr);
static Result<GC::Ref<SourceTextModule>, Vector<ParserError>> parse_from_pre_parsed(FFI::ParsedProgram* parsed, NonnullRefPtr<SourceCode const> source_code, Realm&, StringView filename, Script::HostDefined* host_defined = nullptr);
static Result<GC::Ref<SourceTextModule>, Vector<ParserError>> parse_from_pre_compiled(FFI::CompiledProgram* compiled, NonnullRefPtr<SourceCode const> source_code, Realm&, StringView filename, Script::HostDefined* host_defined = nullptr);
static Result<GC::Ref<SourceTextModule>, Vector<ParserError>> parse_from_bytecode_cache(NonnullRefPtr<RustIntegration::DecodedBytecodeCache>, NonnullRefPtr<SourceCode const> source_code, Realm&, StringView filename, Script::HostDefined* host_defined = nullptr);
LibJS: Add a barebones SourceTextModule class This corresponds to "Source Text Module Record" from the spec.
2021-09-14 15:51:16 -03:00
LibJS+LibWeb: Adopt RootHashTable for unrooted GC-pointer storage
2026-05-19 10:41:17 -03:00
virtual Vector<Utf16FlyString> get_exported_names(VM& vm, GC::RootHashTable<GC::Ref<Module const>>& export_star_set) override;
LibJS+LibWeb+WebContent: Port JS::PropertyKey to UTF-16 This has quite a lot of fall out. But the majority of it is just type or UDL substitution, where the changes just fall through to other function calls. By changing property key storage to UTF-16, the main affected areas are: * NativeFunction names must now be UTF-16 * Bytecode identifiers must now be UTF-16 * Module/binding names must now be UTF-16
2025-08-02 20:27:29 -03:00
virtual ResolvedBinding resolve_export(VM& vm, Utf16FlyString const& export_name, Vector<ResolvedBinding> resolve_set = {}) override;
LibJS: Implement Module linking and evaluating
2022-01-18 15:29:17 -03:00
LibJS: Make Script and Module GC-allocated This ensures that code currently in any active or saved execution stack always stays alive.
2022-09-05 09:31:25 -03:00
Object* import_meta() { return m_import_meta; }
LibJS: Implement import.meta for bytecode
2023-07-12 00:07:12 -03:00
void set_import_meta(Badge<VM>, Object* import_meta) { m_import_meta = import_meta; }
LibJS: Implement Module linking and evaluating
2022-01-18 15:29:17 -03:00
LibJS: Add alternative source-to-bytecode pipeline in Rust Implement a complete Rust reimplementation of the LibJS frontend: lexer, parser, AST, scope collector, and bytecode code generator. The Rust pipeline is built via Corrosion (CMake-Cargo bridge) and linked into LibJS as a static library. It is gated behind a build flag (ENABLE_RUST, on by default except on Windows) and two runtime environment variables: - LIBJS_CPP: Use the C++ pipeline instead of Rust - LIBJS_COMPARE_PIPELINES=1: Run both pipelines in lockstep, aborting on any difference in AST or bytecode generated. The C++ side communicates with Rust through a C FFI layer (RustIntegration.cpp/h) that passes source text to Rust and receives a populated Executable back via a BytecodeFactory interface.
2026-02-23 07:50:46 -03:00
// Pre-computed module declaration instantiation data.
// These are extracted from the AST at construction time so that
// initialize_environment() can run without walking the AST.
struct FunctionToInitialize {
GC::Ref<SharedFunctionInstanceData> shared_data;
Utf16FlyString name;
};
struct LexicalBinding {
Utf16FlyString name;
bool is_constant { false };
i32 function_index { -1 }; // index into m_functions_to_initialize, -1 if not a function
};
LibWeb: Compile remaining script functions in background After off-thread script or module compilation hands top-level bytecode back to the main thread, clone the remaining lazy function payloads and compile them on the thread pool. Install completed bytecode only when the function is still lazy. If the main thread compiled it first, discard the stale result and schedule another pass over that executable so nested lazy payloads still move to bytecode.
2026-05-05 10:02:48 -03:00
Bytecode::Executable* cached_executable() const { return m_executable; }
SharedFunctionInstanceData* top_level_await_shared_data() const { return m_tla_shared_data; }
LibJS: Route bytecode cache installs through owners Store ExecutableBacking on Script and SourceTextModule, and make those owners coordinate generated cache installation. Replacing executable backing now clears non-cache compile inputs and verifies that mapped cache-backed records no longer retain source or heap bytecode inputs.
2026-05-21 16:50:08 -03:00
ExecutableBacking const& executable_backing() const { return m_executable_backing; }
[[nodiscard]] bool can_generate_bytecode_cache() const;
[[nodiscard]] bool can_install_generated_bytecode_cache() const;
void begin_bytecode_cache_generation();
void finish_bytecode_cache_generation_without_install();
LibWeb+LibJS: Cache decoded JS bytecode sidecars Add a ref-counted decoded bytecode cache backing so bytecode cache materialization can create fresh script or module records from a shared decoded sidecar without passing around one-shot raw blob ownership. Keep that backing in ExecutableBacking for records materialized from bytecode cache sidecars, so the immutable decoded data stays alive for as long as the installed record needs it. Cover the shared backing path with a bytecode-cache test that materializes and runs two scripts from one decoded backing.
2026-05-28 14:09:12 -03:00
bool try_install_bytecode_cache(NonnullRefPtr<RustIntegration::DecodedBytecodeCache>, NonnullRefPtr<SourceCode const> source_code);
void install_generated_bytecode_cache(NonnullRefPtr<RustIntegration::DecodedBytecodeCache>, NonnullRefPtr<SourceCode const> source_code);
LibWeb: Compile remaining script functions in background After off-thread script or module compilation hands top-level bytecode back to the main thread, clone the remaining lazy function payloads and compile them on the thread pool. Install completed bytecode only when the function is still lazy. If the main thread compiled it first, discard the stale result and schedule another pass over that executable so nested lazy payloads still move to bytecode.
2026-05-05 10:02:48 -03:00
LibJS: Implement Module linking and evaluating
2022-01-18 15:29:17 -03:00
protected:
LibJS: Remove implicit wrapping/unwrapping of completion records This is an editorial change in the ECMA-262 spec, with similar changes in some proposals. See: - https://github.com/tc39/ecma262/commit/7575f74 - https://github.com/tc39/proposal-array-grouping/commit/df899eb - https://github.com/tc39/proposal-shadowrealm/commit/9eb5a12 - https://github.com/tc39/proposal-shadowrealm/commit/c81f527
2022-05-02 15:54:39 -03:00
virtual ThrowCompletionOr<void> initialize_environment(VM& vm) override;
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
virtual ThrowCompletionOr<void> execute_module(VM& vm, GC::Ptr<PromiseCapability> capability) override;
LibJS: Implement Module linking and evaluating
2022-01-18 15:29:17 -03:00
private:
LibJS: Route bytecode cache installs through owners Store ExecutableBacking on Script and SourceTextModule, and make those owners coordinate generated cache installation. Replacing executable backing now clears non-cache compile inputs and verifies that mapped cache-backed records no longer retain source or heap bytecode inputs.
2026-05-21 16:50:08 -03:00
SourceTextModule(Realm&, StringView filename, Script::HostDefined* host_defined, bool has_top_level_await, Vector<ModuleRequest> requested_modules, Vector<ImportEntry> import_entries, Vector<ExportEntry> local_export_entries, Vector<ExportEntry> indirect_export_entries, Vector<ExportEntry> star_export_entries, Optional<Utf16FlyString> default_export_binding_name, Vector<Utf16FlyString> var_declared_names, Vector<LexicalBinding> lexical_bindings, Vector<FunctionToInitialize> functions_to_initialize, Vector<GC::Root<SharedFunctionInstanceData>> shared_function_data, GC::Ptr<Bytecode::Executable> executable, GC::Ptr<SharedFunctionInstanceData> tla_shared_data, ExecutableBacking);
Libraries: Clean up UTF-16 source text paths Store parser errors, source range filenames, source code filenames, module source, and Rust parser errors as UTF-16 where they flow back into JavaScript-visible strings. Keep byte-oriented source buffers byte-backed. Remove temporary PrimitiveString, ByteString, and UTF-8 detours from JSON, RegExp, module debug logging, print formatting, and tests.
2026-06-21 14:03:19 -03:00
static Result<GC::Ref<SourceTextModule>, Vector<ParserError>> from_rust_result(Optional<Result<RustIntegration::ModuleResult, Vector<ParserError>>>, Realm&, StringView filename, Script::HostDefined*);
LibJS: Add alternative source-to-bytecode pipeline in Rust Implement a complete Rust reimplementation of the LibJS frontend: lexer, parser, AST, scope collector, and bytecode code generator. The Rust pipeline is built via Corrosion (CMake-Cargo bridge) and linked into LibJS as a static library. It is gated behind a build flag (ENABLE_RUST, on by default except on Windows) and two runtime environment variables: - LIBJS_CPP: Use the C++ pipeline instead of Rust - LIBJS_COMPARE_PIPELINES=1: Run both pipelines in lockstep, aborting on any difference in AST or bytecode generated. The C++ side communicates with Rust through a C FFI layer (RustIntegration.cpp/h) that passes source text to Rust and receives a populated Executable back via a BytecodeFactory interface.
2026-02-23 07:50:46 -03:00
LibJS: Make Script and Module GC-allocated This ensures that code currently in any active or saved execution stack always stays alive.
2022-09-05 09:31:25 -03:00
virtual void visit_edges(Cell::Visitor&) override;
LibJS: Account module storage as external memory Report module record vectors, loaded module request attributes, graph loading visited sets, and module filename storage through the GC external memory hook.
2026-05-06 03:21:43 -03:00
virtual size_t external_memory_size() const override;
LibJS: Route bytecode cache installs through owners Store ExecutableBacking on Script and SourceTextModule, and make those owners coordinate generated cache installation. Replacing executable backing now clears non-cache compile inputs and verifies that mapped cache-backed records no longer retain source or heap bytecode inputs.
2026-05-21 16:50:08 -03:00
Vector<SharedFunctionInstanceData*> collect_shared_function_data();
LibWeb+LibJS: Cache decoded JS bytecode sidecars Add a ref-counted decoded bytecode cache backing so bytecode cache materialization can create fresh script or module records from a shared decoded sidecar without passing around one-shot raw blob ownership. Keep that backing in ExecutableBacking for records materialized from bytecode cache sidecars, so the immutable decoded data stays alive for as long as the installed record needs it. Cover the shared backing path with a bytecode-cache test that materializes and runs two scripts from one decoded backing.
2026-05-28 14:09:12 -03:00
void complete_bytecode_cache_install(GC::Ptr<Bytecode::Executable>, GC::Ptr<Bytecode::Executable> top_level_await_executable, NonnullRefPtr<RustIntegration::DecodedBytecodeCache>);
LibJS: Route bytecode cache installs through owners Store ExecutableBacking on Script and SourceTextModule, and make those owners coordinate generated cache installation. Replacing executable backing now clears non-cache compile inputs and verifies that mapped cache-backed records no longer retain source or heap bytecode inputs.
2026-05-21 16:50:08 -03:00
void verify_executable_backing_invariants();
LibJS: Make Script and Module GC-allocated This ensures that code currently in any active or saved execution stack always stays alive.
2022-09-05 09:31:25 -03:00
LibJS: Always allocate ExecutionContext objects on the malloc heap Instead of allocating these in a mixture of ways, we now always put them on the malloc heap, and keep an intrusive linked list of them that we can iterate for GC marking purposes.
2023-11-27 12:45:45 -03:00
NonnullOwnPtr<ExecutionContext> m_execution_context; // [[Context]]
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
GC::Ptr<Object> m_import_meta; // [[ImportMeta]]
LibJS: Always allocate ExecutionContext objects on the malloc heap Instead of allocating these in a mixture of ways, we now always put them on the malloc heap, and keep an intrusive linked list of them that we can iterate for GC marking purposes.
2023-11-27 12:45:45 -03:00
Vector<ImportEntry> m_import_entries; // [[ImportEntries]]
Vector<ExportEntry> m_local_export_entries; // [[LocalExportEntries]]
Vector<ExportEntry> m_indirect_export_entries; // [[IndirectExportEntries]]
Vector<ExportEntry> m_star_export_entries; // [[StarExportEntries]]
LibJS: Implement Module linking and evaluating
2022-01-18 15:29:17 -03:00
LibJS: Pre-compute module declaration metadata from AST Extract the data needed by initialize_environment() from the AST at construction time, following the pattern already used by Script for global declaration instantiation. This pre-computes var declared names, lexical bindings (with function indices), functions to initialize (with SharedFunctionInstanceData), and the default export binding name.
2026-02-11 05:48:00 -03:00
Vector<Utf16FlyString> m_var_declared_names;
Vector<LexicalBinding> m_lexical_bindings;
Vector<FunctionToInitialize> m_functions_to_initialize;
Optional<Utf16FlyString> m_default_export_binding_name;
LibJS: Cache compiled executable and pre-create TLA shared data For non-TLA modules, cache the compiled Bytecode::Executable on the SourceTextModule so we only compile once. For TLA modules, pre-create the SharedFunctionInstanceData for the async wrapper function at construction time. The wrapper function's first invocation will compile it to bytecode and then automatically drop the AST via clear_compile_inputs().
2026-02-11 05:49:52 -03:00
LibJS: Route bytecode cache installs through owners Store ExecutableBacking on Script and SourceTextModule, and make those owners coordinate generated cache installation. Replacing executable backing now clears non-cache compile inputs and verifies that mapped cache-backed records no longer retain source or heap bytecode inputs.
2026-05-21 16:50:08 -03:00
SharedFunctionInstanceDataList m_shared_function_data;
LibJS: Cache compiled executable and pre-create TLA shared data For non-TLA modules, cache the compiled Bytecode::Executable on the SourceTextModule so we only compile once. For TLA modules, pre-create the SharedFunctionInstanceData for the async wrapper function at construction time. The wrapper function's first invocation will compile it to bytecode and then automatically drop the AST via clear_compile_inputs().
2026-02-11 05:49:52 -03:00
GC::Ptr<Bytecode::Executable> m_executable;
GC::Ptr<SharedFunctionInstanceData> m_tla_shared_data;
LibJS: Route bytecode cache installs through owners Store ExecutableBacking on Script and SourceTextModule, and make those owners coordinate generated cache installation. Replacing executable backing now clears non-cache compile inputs and verifies that mapped cache-backed records no longer retain source or heap bytecode inputs.
2026-05-21 16:50:08 -03:00
ExecutableBacking m_executable_backing;
LibJS: Add a barebones SourceTextModule class This corresponds to "Source Text Module Record" from the spec.
2021-09-14 15:51:16 -03:00
};
}
Reference in a new issue Copy permalink