Produce JS-visible string results as UTF-16 at their source, including
numeric formatting, BigInt and BigFraction formatting, URI encoding,
console formatting, parser errors, regular expression errors, Intl and
Temporal records, LibUnicode locale boundaries, and LibWeb bindings.
Handle fractional radix formatting through the UTF-16 builder view.
Move the remaining LibJS primitive string users to UTF-16 views and
strings. Remove the primitive string UTF-8 accessors and byte-string
coercion paths so new callers cannot rely on the old storage model.
The Storage panel expects storesUpdate messages after watched storage
changes. Summarize committed IndexedDB mutation logs into DevTools paths
and forward them through WebContent so Firefox can refresh database,
object-store, and record rows without polling.
Firefox's behaviour is less than ideal here. A lot of things don't
update automatically even inspecting a page in Firefox. Some
things (like new databases) won't show up until you fully refresh the
page. So that makes it a bit hard to know that we're doing things
correctly. As far as I can tell, we are at least behaving as well as
Firefox requires.
We do have one workaround: Firefox doesn't display record updates
without a manual refresh, and in fact any change messages for them show
up as rows in the host's database table. So for now, we filter them out
to avoid visual weirdness in the inspector.
DevTools needs to inspect IndexedDB state from WebContent, but the
storage registry and object store data live behind LibWeb internals.
Expose a small inspection API that can enumerate IndexedDB hosts,
databases, object stores, and records without making DevTools know about
the internal database objects directly.
Enable -Wexit-time-destructors for all in-tree library targets and
update process-lifetime library statics so they no longer register
exit-time destructors. Long-lived caches, lookup tables, singleton
registries, and generated constants now use NeverDestroyed or leaked
references where the data is intended to live until process exit.
Update LibWeb, LibLine, and the binding generators so regenerated
sources follow the same rule instead of reintroducing destructed
statics.
Represent BufferSource and ArrayBufferView as ordinary IDL typedefs over
their underlying union types, instead of special casing in the IDL
generator. This allows the union conversion/return machinery handle
these types consistently with other typedefs, which removes buffer
specific paths from the IDL generator.
This necessitates changing the WebIDL::BufferSource and
WebIDL::ArrayBufferView classes as views over these variants. This
replaces the old GC backed BufferableObject wrapper structure and
provide convenience helpers to determine things such as the byte length,
byte offset, backing buffer, and typed-array APIs.
Represent WebIDL C++ types with a single CppType model that tracks
nullability, optional presence, and contained storage.
GC-like values now use GC::Ref/GC::Ptr directly, while containers choose
"plain", "Root", or "Conservative" container types depending on what
they contain. For example, sequence<Element> becomes a RootVector of
GC::Ref values, while sequence<SomeDictionary> becomes a
ConservativeVector only when the dictionary contains GC-like values.
This moves the generated bindings away from wrapping GC values in
GC::Root by default.
This has broad fallout as the types passed to interfaces for GC
objects changes almost fully across the board.
Similar to GC::Root<T>, make GC::RootVector<T> constructible without
explicitly passing a Heap.
This is implemented by having RootVectorBase use GC::Heap::the() for
heap-free construction.
The process-wide IndexedDB database registry stores internal database
objects in GC roots. Those objects do not need a JavaScript shape or
realm, but they inherited one from PlatformObject. This let an internal
database keep its creating realm, window, and script graph alive.
Make the internal database construct a plain GC cell instead. Callers
provide a heap when they need snapshots of associated connections. This
means the database no longer needs to remember its creating realm.
On x.com, the rooted internal database subgraph shrank from the whole
page graph, including 24k executables, to the database metadata itself.
Keep index records ordered as they are stored instead of appending and
sorting the entire record vector for every insert. This avoids repeated
O(n log n) work during bulk object-store writes while preserving the
ordering by index key and referenced object-store key.
Add a text test covering out-of-order index writes with repeated index
keys, so cursor iteration still observes the required record order.
Previously we were inconsistent by generating code for enum definitions
but not generating code for dictionaries. With future changes to the
IDL generator to expose helpers to convert to and from IDL values
this produced circular depdendencies. To solve this problem, also
generate the dictionary definitions in bindings headers.
Previously, the LibWeb bindings generator would output multiple per
interface files like Prototype/Constructor/Namespace/GlobalMixin
depending on the contents of that IDL file.
This complicates the build system as it means that it does not know
what files will be generated without knowledge of the contents of that
IDL file.
Instead, for each IDL file only generate a single Bindings/<IDLFile>.h
and Bindings/<IDLFile>.cpp.
Without this, an open request could hang if a prior connection was not
explicitly close()d but instead allowed to go out of scope.
The spec says that the dangling connection should be closed when the
execution context it was opened in is destroyed, but no other browser
does so. Detecting the execution context being destroyed would likely
mean a lot of overhead in JS calls, so it's best to avoid that, despite
this being observable.
Wrap the SerializationRecord (Vector<u8, 1024>) in an OwnPtr so that
each ObjectStoreRecord is only ~16 bytes instead of ~1040+ bytes.
This makes Vector operations on the records list dramatically cheaper
since memmove now shifts pointers instead of kilobyte-sized buffers.
Since records are sorted by key, records matching a key range form a
contiguous block. Use binary search to find the range boundaries and
remove the block in one operation, instead of scanning every record
with is_in_range().
Instead of appending and re-sorting the entire records vector on every
insert (O(n log n)), use binary search to find the correct insertion
position and insert directly (O(log n) comparisons + O(n) shift).
If one request on a transaction succeeds, then the next one fails, that
would cause the abort algorithm to run before the success for the first
request due to the task queue ordering. Instead, queue the processing
for the next request after the completion of the current request.
This code was treating the parameter to the JS::Array as if it was a
capacity, but that causes JS to see more databases than are actually
accessible if a database was being created in an aborted transaction.
To allow these to be reverted, we store mutation logs per object store
in the scope of a readwrite transaction to track the modifications that
were made by it. If a revert is needed, the log is played in reverse to
bring us back to the original state.
These should persist for the duration of the program, at least until we
have persistent storage to restore from if a database needs to be used
again.
This fixes a flake in indexeddb-queued-delete-after-open that turns
into a consistent failure (or an assertion failure in GC::Weak in
debug builds) when running the test with the -g flag.
cleanup_indexed_database_transactions() is called on every event
loop spin. It was calling associated_connections() which allocates
a GC::HeapVector every time, creating massive GC pressure on
JS-heavy sites (217K allocations observed on x.com).
Switch the hot path to use a GC::RootVector instead, which lives
on the stack and avoids GC heap allocation while still being
visible to the garbage collector.
Rename the existing methods to make the return type explicit:
- associated_connections_as_heap_vector() for callers that
capture the result in GC::Function lambdas
- associated_connections_as_root_vector() for callers that
just iterate safely within a single scope
The database map stores GC::Weak<Database> entries. When the GC
collects a Database, the weak pointer goes null but the map entry
remains. The old code dereferenced the weak pointer without checking
liveness, causing a null reference binding (UBSan).
Fix this by checking ptr() before dereferencing, and cleaning up the
stale map entry if the database was collected.
...rather than at each of the open connections. These disagreed with the
spec steps written right above them.
The transaction-lifetime.any.html and idbdatabase_close.any.html tests
pass instead of timing out with these changes, so they've been imported.
If the current JS task has not made any requests, then nothing else
will trigger a commit like the spec desires, so we need to do it in the
microtask checkpoint.
Two WPT tests no longer time out with this change and have been
imported.
If an error causes this to be left inactive, further requests will be
rejected on the transaction. This gives us a few subtest passes in
IndexedDB/key-conversion-exceptions and IndexedDB/keypath-exceptions
WPTs.
With this fixed, the cleanup loop can assert that the transactions'
states are all active before they are set inactive.
This fixes the regression in idbindex_reverse_cursor.any.html, which
was actually exposing the underlying issue of ignoring conflicting
read/write transactions. Now, if a read/write transaction is in the
queue, no transactions can coincide with its requests' execution.
Previously, after one request was marked as processed, we would
synchronously queue another task to process the next request. This
would mean that two open requests on the same database could
interleave. This was especially problematic when one of the requests
would cause the database to upgrade, since the second open request
would begin processing before the upgradeneeded event fired, causing an
exception to be thrown in the second open().
The solution is to explicitly check for continuation conditions after
events have been fired in order to ensure that every step for the
request is completed before starting any further request processing.
For connection requests, the spec states:
> Open requests are processed in a connection queue. The queue contains
> all open requests associated with an storage key and a name. Requests
> added to the connection queue processed in order and each request
> must run to completion before the next request is processed. An open
> request may be blocked on other connections, requiring those
> connections to close before the request can complete and allow
> further requests to be processed.
For requests against a transaction, the spec states:
> Once the transaction has been started the implementation can begin
> executing the requests placed against the transaction. Requests must
> be executed in the order in which they were made against the
> transaction. Likewise, their results must be returned in the order
> the requests were placed against a specific transaction. There is no
> guarantee about the order that results from requests in different
> transactions are returned.
In the process of reworking it to use this approach, I've added a bunch
of new tests that cover things that our imported WPTs weren't checking.
With the fix for serializing connection requests, we can now fully
download the assets for the emscripten-compiled asm.js games in the
Humble Mozilla Bundle, particularly FTL: Faster Than Light.
There were no regressions in our test suite. One web platform test,
'idbindex_reverse_cursor.any.html', has one newly-failing subtest, but
the subtest was apparently only passing by chance due synchronous
execution of requests. A few web platform tests that were added in a
prior commit improved. The delete-request-queue.any.html test has
stopped crashing, and the close-in-upgrade-needed.any.html test has
stopped flaking, so they are both imported here as well.
Incidentally fixes#7512, for which a crash test has been added.
This reduces it to one GC::Root instead of one per element, but also
will make it easier to replace that Root with a Ref when needed in the
next commit.
