Snapshot DevTools DOM mutation payloads immediately, but defer subtree
serialization until the target document's layout is up to date.
This avoids re-entering layout-backed DOM serialization from style and
layout updates, which could crash WebContent while DevTools was
listening for mutations.
This exposes the operation through RequestServer.ipc and RequestClient
so the client can create the stub before accessing a content-keyed
side-data file.
This lays the groundwork to allow shared workers, nested workers (i.e.
workers owned by workers) and service workers to function independently
of WebContent.
The compositor present path submitted Skia work with a synchronous GPU
wait, so each present_frame call stayed blocked until the backend
finished the submitted work. That serialized compositor processing
behind GPU completion even though the presented-bitmap acknowledgement
already controls backing-store reuse.
Add an async Skia flush-and-submit entrypoint and have the compositor
track pending presents until the Skia finished callback runs back on the
compositor event loop. did_present_frame now fires from that completion
path, while the existing presented_bitmap_ready_to_paint acknowledgement
remains the reuse gate for client-presented bitmaps.
Ganesh does not reliably run the finished proc without explicit async
work polling on this backend, so pending presents keep a completion
timer alive to call checkAsyncWorkCompletion().
Track whether WebContent still needs a beforeunload check and let the
frontends immediately remove a tab or window when no prompt can be
shown. WebContent still receives the close request so pagehide, unload,
and cleanup steps can run.
When the visible view is removed immediately, keep detached ownership of
the WebContent page until it reports that the top-level traversable
closed. If no acknowledgement arrives, release detached ownership and
ask ProcessManager to shut down the unused WebContent process.
Compositor presentation still preserved the old implicit replay flush
and then submitted again after viewport scrollbar overlay painting. That
left a redundant flush on every frame and an extra GPU submit when
overlay scrollbars were painted.
Flush the display list player once after compositor overlay painting in
both presentation and screenshot paths. The explicit flush now covers
all compositor painting for the target surface, so direct surface
flushes are no longer needed.
DisplayListPlayer::execute() used to flush the active painting surface
as part of replay. That made replay and submission inseparable, so
callers could not add extra painting after replay without either
accepting a stale submission boundary or flushing again.
Expose flush(PaintingSurface&) on the player and make execute() only
replay commands. Existing callers now issue an explicit flush at the
same point where the implicit flush used to happen, keeping behavior
unchanged while making the submission boundary visible to compositor
code.
Refresh Qt chrome palettes from the current color scheme when the
application palette changes. Recompute autocomplete and location field
colors and repaint WebContent immediately so theme changes do not
require another interaction or restart.
Input events can arrive after a page has been removed during browser
teardown. Those events used to be queued anyway, but no page could
request a frame to drain the queue and send the matching input-event
completion response back to LibWebView.
Drop input events whose target page no longer exists and immediately
notify the client with EventResult::Dropped. The mouse-event coalescing
path now uses the page lookup performed before enqueueing, avoiding the
diagnostic for missing pages while keeping valid events on the normal
queue path.
Browser UI input can race with compositor context teardown during tab
close. The control connection may ask the compositor to handle, scroll,
or forward a mouse event after WebContent has already destroyed the
context. Treating that as an invariant crashes the compositor even
though the event is stale.
Make the UI-facing mouse handling and async scroll entry points return
false when the context is gone. Mouse forwarding now reports whether it
actually dispatched the event, letting LibWebView fall back to direct
WebContent IPC and preserve the normal input-event acknowledgement flow.
The WebContent compositor host is now always backed by the compositor
process, so the private ProcessWebContentCompositorHost name no longer
distinguishes it from another implementation.
Rename the internal class to WebContentCompositorHost while keeping the
existing factory API unchanged.
The delayed backing-store shrink policy was still owned by the LibWeb
compositor context handle. That made the handle cache viewport state and
own a Core::Timer even though backing-store allocation now lives in the
compositor service.
Move the resize-completion timer into CompositorState::ContextState,
next to the viewport state and BackingStoreManager. The compositor
service still pads backing stores while a top-level resize is in
progress, then flips the context back to a non-resizing state after the
same delay so the stores can shrink. LibWeb now just forwards
viewport-size updates through CompositorHost.
CompositorHost still accepted page presentation metadata so the old
in-process compositor path could register new contexts while
constructing the LibWeb handle. With the compositor process as the only
backend, context allocation and service registration already happen
before LibWeb creates the handle.
Make CompositorHost::create_context() take only the allocated context id
and remove the private register_context() hook. Navigable no longer
computes page metadata for the host, and WebContent no longer carries a
erification-only registration override.
The compositor thread used to need a way for LibWeb to postpone
adopting async scroll offsets when the thread had already presented
newer state. The compositor process implementation never defers this
path, so the hook became a hardcoded false result after the thread code
was removed.
Remove the host callback and let Navigable always consume the pending
async scroll updates from the compositor process before running
rendering-update observers.
Compositor process is now the only supported compositor topology, so the
old in-process compositor thread and its WebContent-local IPC bridge are
dead code.
Remove the command-line option that allowed Browser to start
WebContent with the in-process compositor thread instead of the
Compositor helper process. The compositor process is now the default
path, so the opt-out flag and the matching WebContent selector only kept
the old thread mode reachable.
With that option gone, simplify Browser and WebContent startup to wire
each WebContent process to the Compositor process directly. The old
local Browser-to-WebContent compositor IPC setup and its fallback input
and ready-to-paint branches are no longer needed.
Browser-side viewport-size updates can race with WebContent context
teardown. Other late presentation callbacks already tolerate a missing
context after destruction, but viewport updates still asserted that the
context exists and could bring down the compositor during parallel
test-web runs.
Treat missing contexts on this path as stale updates and drop them. The
WebContent-owned paths still verify context ownership before mutating
compositor state.
Notify the UI process with the solid canvas background color recorded
for the top-level document. This is the Canvas system color with the
effective document background composited over it, matching the color
used before normal painting.
Store that color on the view and use it when AppKit, Qt, and Gtk need
to fill areas exposed while an older bitmap is still on screen during a
window resize.
Replace the local substring matcher with the adblock-rust engine exposed
through the dedicated content blocker Rust FFI. Keep the previous engine
when a replacement list cannot be parsed.
Pass shared list buffers directly into Rust instead of building a
duplicate C++ vector of lines first. Generate cosmetic CSS through the
Rust matcher, including generic class and id selectors collected from
shadow-including descendants.
Keep a supplemental index for generic cosmetic selector-list rules.
adblock-rust indexes these rules under the first class or id token only,
so also key them by later simple class and id selectors in the list.
Update ContentBlocker coverage for rule options, exceptions, third-party
checks, blob and file URLs, invalid list replacement, filtering toggles,
cosmetic CSS, and generic selector-list cosmetics.
The browser previously treated the out-of-process Compositor as fatal.
Restart the shared Compositor from the browser process, reconnect
process-backed WebContent clients, recreate compositor contexts, restore
viewport state, and ask WebContent to repaint and republish canvas and
media resources. WebContent now marks its compositor connection lost,
returns conservative values for synchronous compositor queries while
reconnecting, and drops outgoing updates until the replacement transport
arrives.
Synchronous input queries through the compositor control connection now
use fallible IPC. If the Compositor exits after the open check or before
the sync reply arrives, scroll and mouse handling report that the
Compositor did not handle the event and let the normal WebContent
fallback run.
Mouse events queued while the Compositor is unavailable now fall back to
direct WebContent dispatch. This keeps input completion in step with the
pending-event queue.
Recovery is capped at three automatic restarts. If the restart limit is
exceeded, if restart, reconnect, or context recreation fails, or if the
replacement Compositor exits during active recovery, the browser crashes
instead of switching process-backed views to a fallback path.
Represent WebIDL C++ types with a single CppType model that tracks
nullability, optional presence, and contained storage.
GC-like values now use GC::Ref/GC::Ptr directly, while containers choose
"plain", "Root", or "Conservative" container types depending on what
they contain. For example, sequence<Element> becomes a RootVector of
GC::Ref values, while sequence<SomeDictionary> becomes a
ConservativeVector only when the dictionary contains GC-like values.
This moves the generated bindings away from wrapping GC values in
GC::Root by default.
This has broad fallout as the types passed to interfaces for GC
objects changes almost fully across the board.
We are moving toward an architecture where the browser owns a single
process that holds the GPU context, so Skia resources can be shared
across every renderer and WebContent processes can be sandboxed away
from direct GPU access. A dedicated Compositor helper process is the
first step. The earlier commits laid the foundation; this one turns the
helper on as a selectable backend, gated behind
--enable-compositor-process so the existing in-process path stays the
shipping default.
Default topology -- one compositor per WebContent, in-process:
+---------+
| Browser |
+---------+
/ | \
v v v
+---+ +---+ +---+ each WebContent runs its own
|WC | |WC | |WC | CompositorThread on a dedicated
|+C+| |+C+| |+C+| thread, with its own GPU context.
+---+ +---+ +---+
Opt-in topology -- one single-threaded Compositor, shared by all WCs:
+---------+ control +-------------------+
| Browser |<----------->| Compositor |
+---------+ | (single thread, |
/ | \ | single GPU ctx, |
v v v data | shared by all |
[WC] [WC] [WC] ---------->| connected WCs) |
+-------------------+
Three IPC channels carry the work in the opt-in topology. The
existing Browser<->WebContent channel gains context allocation.
A new Browser<->Compositor channel carries context lifetime,
viewport, UI input, and presentation acks plus backing-store and
frame upcalls. A new WebContent<->Compositor channel carries
display lists, scroll state, video, compositor surfaces, async
scrolling, presentation, and screenshots, with upcalls for
delegated input and compositor loss.
The helper process is a Browser-owned singleton, so Browser has to be
the source of truth for which contexts exist: it is the only party that
can register them with the helper, route backing-store and presentation
upcalls back to the right WebContent and page, and reap them when a
WebContent crashes. Move id allocation into Browser and have WebContent
ask for ids over IPC, so the runtime switch in the next commit only has
to select the backend. Behavior is preserved because Browser reproduces
the previous deterministic-vs-fresh allocation logic and the in-process
compositor still owns rendering.
With CompositorState in place, Browser and WebContent need an actual
wire protocol to drive it: which calls are sync, who validates that a
WebContent is allowed to touch a given context, how compositor loss
propagates back to the renderer. This commit fills in the four endpoints
around CompositorState and the matching WebContent-side helpers so the
future remote host can forward CompositorHost calls directly. The
service still has no clients, so default rendering is unchanged.
The helper process needs somewhere to hold the per-context state
CompositorThread currently keeps inside WebContent — display list
and resources, scroll state, async scroll tree, video and compositor
surfaces, backing stores, presentation bookkeeping — before any incoming
IPC has something to dispatch into. CompositorState supplies that, with
two small client interfaces that the next commit plugs the Browser- an
WebContent-side actors into. A few LibWeb painting helpers are exported
with WEB_API so the service can link without dragging in the rest of
the painting code. Nothing references the new class yet.
Once the compositor lives in another process, the helper has to know a
context's id before any per-context message about it can be dispatched.
Today the id is minted inside CompositorHost::create_context and
returned to the caller, so it cannot be named ahead of time.
Untangle allocation from creation so Browser can mint the id and hand it
down through the call chain into both the local host and the future
remote host with no special case. The id helpers also move into a public
header so LibWeb, WebContent, and the upcoming service share one
encoding for the page-presenting bit. Behavior is preserved; the
in-process compositor still owns rendering.
The compositor is moving into a dedicated helper process. That requires
a process to launch, channels for Browser and WebContent to talk to it
over, and client proxies on each side. Land all of that as an inert
scaffold first, gated behind --enable-compositor-process, so the default
rendering path is unchanged and later commits can fill in the protocol,
the service-side state, and the runtime switch against a stable target.
Add a repeatable blocker-list option that reads local list files in the
browser process. The files are concatenated into one buffer and shared
with WebContent through the content blocker IPC path when view options
are applied.
Parse the buffer in WebContent and reject malformed UTF-8 without
replacing the currently installed rules.
Split cosmetic blocker rules out from network patterns. Expose matching
rules as user CSS through StyleScope.
Invalidate affected user style caches when blocker state changes.
Generated cosmetic CSS now respects disabled content blocking.
Rename the local content blocking implementation and its tests from
ContentFilter to ContentBlocker while keeping the existing substring
matcher backend and behavior.
Update the WebContent IPC method, WebView option names, debug toggle,
and default config file name to use content blocker terminology.
Most of this functionality was already implicitly disallowed for
element-reference pseudo-elements by the fact that we weren't creating
entries in `m_pseudo_element_data` for them, but we need to explicitly
limit it in preparation of creating those entries.
Due to the above this is mostly non-functional apart from a regression
where we no longer support custom properties on element-reference
pseudo-elements. Previously when setting custom properties for an
element-reference pseudo-element we would call `ensure_pseudo_element()`
which created a synthetic pseudo-element entry distinct from the
referenced element which only stored custom property data - this was
clearly wrong and will be implemented properly in a future commit.
LibWeb still exposed the concrete CompositorThread to Page,
Navigable, and EventHandler, so compositor IPC would have leaked the
thread implementation into callers. The old thread APIs also bundled
page presentation callbacks and main-thread wakeups into the same
object, which made it awkward for WebContent to put an actor boundary
in between.
Introduce CompositorHost and context handles as the caller-facing API,
and move shared compositor protocol values out of CompositorThread. Add
WebContentCompositor IPC endpoints and route PageHost through a paired
in-process transport. The actor owns CompositorThread with explicit
main-thread and UI presentation clients, while screenshot completion is
serialized on the WebContent event loop using request IDs.
The intention for introducing IPC here is to prepare for moving the
compositor thread into a separate process.
Similar to GC::Root<T>, make GC::RootVector<T> constructible without
explicitly passing a Heap.
This is implemented by having RootVectorBase use GC::Heap::the() for
heap-free construction.
Load the browser-generated crash page as a synthetic response for the
URL that was active when WebContent exited. This keeps the session
history entry, response URL, and created Document aligned with the same
navigation URL, so reload targets the original page without creating a
local document for an HTTP(S) history entry.
Suppress history metadata updates from the generated page and declare
an inert rel=icon. Fallback favicon loading now follows the HTML
condition that no link with the icon keyword exists, which avoids the
credentialed /favicon.ico request from the crashed origin.
Add Core::TimeZone as the single entry point for changing the
current time zone. This updates both ICU and the process TZ/tzset
state so libc-backed helpers such as AK::UnixDateTime::to_string()
agree with JS/ICU time zone state. Previously only the ICU timezone
would be updated, which could result in inconsistent results being
returned.
Move compositor-thread ownership to WebContent's PageHost so every Page
object in one WebContent process registers its navigables on the same
compositor thread. This covers auxiliary pages created by window.open(),
while worker and SVG helper pages continue to skip compositor thread
creation.
Keep page presentation keyed by page id. Each presenting context records
the page id it presents for, and static compositor entry points route
ready-to-paint, async scrolling, and viewport scrollbar events to that
page's presenting context on the shared thread.
BackingStoreManager was owned by Navigable and allocated on the GC heap,
which left backing-store sizing decisions on the main thread even though
the compositor thread was already responsible for allocating the actual
surfaces. That split ownership makes it harder to isolate the compositor
behind a process boundary.
Move the manager into LibWeb's compositor code and let CompositorThread
thread data own it. The main thread now reports viewport size changes
through a compositor command, and the compositor uses that message to
decide when to resize and publish backing stores. The delayed shrink
timer remains with the main-thread CompositorThread facade so it can use
the Core event loop, but it now only sends another viewport-size update.
This removes the GC edge from Navigable, drops stale WebContent includes
and keeps the existing resize padding and delayed shrink behavior.
Map JavaScript bytecode cache sidecars from the HTTP disk cache instead
of copying them into anonymous shared buffers while handing response
headers to WebContent. Store the mapped data as ImmutableBytes on the
fetch response so script fetching can decode directly from the mapped
sidecar bytes.
Add LibHTTP coverage for retrieving associated cache data as a mappable
file, alongside the existing byte-buffer retrieval API.
Let CacheEntryWriter expose an explicit body-file handoff after a
successful flush. RequestServer sends that file to WebContent before the
request finishes, so Fetch can replace its retained memory-cache buffer
with a file-backed ImmutableBytes body.
Keep the plain flush path fd-free for existing callers, and add coverage
for mapping the body file returned by the writer. The focused disk and
memory cache web tests continue to pass.
Expose validated disk cache entry bodies as fd-backed byte ranges. Use
this for cache hits instead of creating a response socket.
Map received body ranges into ImmutableBytes on the client side. Keep
the streamed path for network responses. Adapt buffered callbacks to
receive ImmutableBytes, so cached responses can stay file-backed.
RequestServer, LibRequests, and LibWebView build with the new IPC
message and callback type.
Store source map locations as bytecode offset, line, and column.
Runtime consumers only emit the start line and column, so source end
positions and source text offsets do not need to be carried through
Executable source maps, bytecode cache serialization, or the Rust FFI.
Keep SourceCode's internal position cache able to track source text
offsets so callers can still translate source offsets to line and
column pairs when needed. Hash dump-bytecode IDs from the name, first
source position, and bytecode size instead of source slices that need
end offsets.
Bump the bytecode cache format version for the slimmer serialized
source map entry shape.
Route mouse events over the compositor IPC path while async scrolling is
enabled, and let the compositor capture viewport scrollbar drags before
the main-thread event path sees them. Dragging mutates the compositor
scroll tree and schedules an async-scroll present, so the viewport thumb
and content can move without a main-thread round trip.
Keep normal main-thread event handling for misses and preserve the wheel
bypass behavior for page events.
The synchronous wheel path treats wheel deltas as CSS-pixel scroll
distances, while the async compositor scroll tree mutates scroll state
stored in device pixels. Passing the same unscaled delta into the
compositor made async scrolling advance too little whenever
device pixels per CSSPixel was greater than 1, so scrolling felt slower
than with async scrolling disabled.
Convert wheel deltas before crossing the compositor boundary and make
the compositor IPC carry only the device-pixel position and delta it
needs. This keeps AsyncScrollTree device-pixel native and makes async
viewport scrolling match the synchronous path across high-DPI displays
and page zoom levels.
Exit WebContent immediately when either browser-side IPC peer
disconnects. Plumb Unix process exit status through LibWebView so the
browser process can tell clean owner-driven shutdown apart from renderer
crashes.
This keeps nonzero exits and signal deaths reported as crashes, while
letting status 0 exits disappear without making test-web report the page
as crashed.
Trace the compositor state transitions that matter once presentation and
async scrolling share one path: page registration, backing-store swaps,
did_paint delivery, ready_to_paint acknowledgements, wheel admission,
and deferred presents waiting for reusable bitmaps.
Keep the logs under COMPOSITOR_DEBUG and use the [Compositor] prefix
consistently. The trace is about ownership and backpressure instead of
dumping every wheel event detail, so stalls can be diagnosed without
reintroducing noisy async-scrolling-only logging.
Send the experimental UI-process wheel bypass to WebContent over the
Compositor IPC connection instead of adding a separate async scrolling
transport. This keeps bypass admission on the channel that owns bitmaps
and ready_to_paint acknowledgements.
If the compositor rejects the wheel, LibWebView falls back to regular
input delivery. If it accepts, mark the queued DOM event as having
already performed its default action so dispatch preserves ordering
without scrolling twice.
Use the snapshot from the previous commit to let CompositorThread apply
experimental viewport wheel deltas when async scrolling is enabled. The
event handler first performs synchronous admission on the main thread,
then enqueues a compositor scroll command instead of mutating live
document scroll state directly.
Rasterize accepted scrolls through the same compositor presentation
path added earlier. The compositor stores the newest async viewport
offset so the next main-thread display-list recording can adopt it
before repainting, preventing older paints from snapping the visible
position backward.
Keep DOM wheel dispatch on the main thread. When the compositor already
performed the default action, dispatch the wheel as non-cancelable and
suppress a second default scroll. Non-viewport targets, nested
scrollers, and pages with blocking wheel listeners stay synchronous.
Introduce a dedicated Compositor IPC channel between the UI process and
WebContent. Use it for backing-store setup, presented bitmap delivery,
and bitmap-specific ready_to_paint acknowledgements.
This makes CompositorThread the single owner of frame presentation
bookkeeping before async scrolling starts producing frames without the
main thread.
Remove old paint and backing-store messages from WebContentClient and
PageClient so the UI process no longer observes two presentation
protocols.
Move HeapTimer out of LibWeb and into LibGC as GC::Timer, inheriting
from GC::Cell instead of JS::Cell. Add a finalize() override that
stops the timer, ensuring it is cleaned up during GC finalization.
Previously we were inconsistent by generating code for enum definitions
but not generating code for dictionaries. With future changes to the
IDL generator to expose helpers to convert to and from IDL values
this produced circular depdendencies. To solve this problem, also
generate the dictionary definitions in bindings headers.
This commit splits out synchronization primitives from LibThreading into
LibSync. This is because LibThreading depends on LibCore, while LibCore
needs the synchronization primitives from LibThreading. This worked
while they were header only, but when I tried to add an implementation
file it ran into the circular dependency. To abstract away the pthread
implementation using cpp files is necessary so the synchronization
primitives were moved to a separate library.
The Paintable tree and its supplemental painting data structures were
GC allocated because that was the easiest way to manage it and avoid
leaks introduced by ref cycles. This included the Paintable subclasses
themselves plus StackingContext, ChromeWidget, Scrollbar, ResizeHandle,
and scroll-frame state.
We are now trying to reduce GC allocation churn on layout and painting
updates, so keeping this short-lived rendering tree outside the JS heap
is a better fit. Move Paintable to RefCountedTreeNode, make painting
helpers ref-counted or weakly reference Paintables, and update the
layout and event-handler call sites to use RefPtr/WeakPtr ownership.
Attach cached JavaScript bytecode sidecars to HTTP response headers so
WebContent can materialize classic and module scripts directly from a
decoded cache blob on cache hits.
Carry the disk cache vary key with the sidecar and reuse it when storing
fresh bytecode, avoiding mismatches against the augmented network
request headers used to create the cache entry.
Keep CORS-filtered module responses intact for status, MIME, and script
creation checks. Read bytecode sidecar data only from the internal
response, and treat decode or materialization failure as a cache miss
that falls back to normal source compilation.
Schedule JavaScript bytecode cache generation after downloaded classic
scripts and modules have been handed back to the main thread.
The cache job reparses and fully compiles on the thread pool,
serializes the bytecode blob, and stores it as HTTP cache sidecar data.
RequestServer finalizes the disk cache entry before notifying
WebContent, so the script fetcher can attach the sidecar immediately.
Add RequestServer messages and LibRequests helpers for storing and
retrieving HTTP cache sidecar data.
Transfer sidecar payloads as anonymous buffers so large JavaScript
bytecode blobs do not have to be serialized into IPC message bodies.
Add the same connect timeout used by regular requests to curl-backed
websockets, buffer unsent websocket bytes, and resume them from a write
notifier instead of retrying curl_easy_send() in a tight loop.
This prevents CPU spins on CURLE_AGAIN and keeps large websocket sends
from hanging when the peer applies backpressure.
Convert DNS failures and connecting-stage close requests into terminal
websocket state changes, and remove websocket entries once they close.
This keeps clients from getting stuck in CONNECTING and makes early
cancelled or failed websocket attempts terminate like the other close
paths.
The Take Screenshot and Take Element Screenshot endpoints add a callback
to the animation frame driver and rely on it firing to encode and return
the framebuffer. Since rendering updates moved to on-demand scheduling,
that callback never runs unless something else happens to request a
frame, so the WebDriver client times out waiting for the response.
Request a frame after registering the callback, matching what
Window::request_animation_frame() does for the JS-exposed API.
Add --dump-style-invalidation-counters=N to Ladybird and propagate it
to WebContent helper processes.
When enabled, WebContent dumps the current document style invalidation
counters with dbgln() after every N recorded style invalidations. This
makes it possible to collect the counters while browsing without adding
temporary C++ logging.
Previously a fixed-rate paint refresh timer kept queueing rendering
update tasks at the maximum configured frame rate, regardless of whether
anything had actually changed. This wasted CPU on idle pages, which
spend most of their time in a steady state where no style, layout, or
paint work is needed.
Replace the repeating timer with a single-shot frame timer driven by
PageClient::request_frame(). A rendering update is now scheduled only
when something requires one. The configured maximum frame rate is
preserved as a ceiling on how closely consecutive frames can follow each
other.
Fixes flakiness in worker tests that create a Worker or SharedWorker
with a missing script URL and only attach an error handler to it.
Once the test callback returns, nothing keeps the worker rooted from
JavaScript. If GC ran before the WebWorker process reported the
script fetch failure, the Worker/WorkerAgentParent cycle could be
collected and the error event never delivered, leaving the test hung
until timeout.
Hold startup-pending WorkerAgentParents from the outside
EnvironmentSettingsObject and release that edge once the script load
succeeds, fails, or the worker closes. The worker now survives long
enough to deliver its first script-load result.
handle_connect_state (used by <link rel=preconnect>) attached easy
handles to the multi without setting CURLOPT_RESOLVE, so libcurl spawned
its own threaded resolver for them. When the system stub resolver was
slow, the thread got stuck and the next ~Request pthread_join'd it on
the main thread for many seconds.
Route Connect-mode requests through the same DNS + CURLOPT_RESOLVE path
that Fetch uses, so libcurl never spawns the thread.
Per-request and per-connection logging that surfaces enough detail
to diagnose where time goes when a page load misbehaves. Gated by a
new REQUESTSERVER_WIRE_DEBUG cmakedefine.
Documentation/RequestServerWireLogging.md describes each label
(wire/wire+/wire++/wire^, wire-batch, wire-stall, wire-burst,
wire-pipe-pressure, LibDNS wire-dns, UI wire-cookie) and how to read
them.
Previously, `AnonymousBuffer::create_with_size(0)` returned an error
because POSIX `mmap` rejects a zero length with `EINVAL`, and Windows
`CreateFileMapping` rejects a zero maximum size for an anonymous
mapping. This caused a crash when using `--headless=text` with zero
size pages like `about:blank`.
write_queued_bytes_without_blocking() used to allocate a Vector sized
to the entire queued response buffer and memcpy every queued byte into
it, on every curl on_data_received callback. When the client pipe was
slower than the network, the buffer grew, and each arriving chunk
triggered a full copy of everything still queued. With enough pending
data this added up to tens of gigabytes of memcpy per request and
stalled RequestServer for tens of seconds.
Drain the stream in a loop using peek_some_contiguous() + send()
directly from the underlying chunk, and discard exactly what the
socket accepted. No intermediate buffer, no copy. The loop exits on
EAGAIN and enables the writer notifier, matching the previous
back-pressure behavior.
When curl invokes the socket callback to tell us it only wants one
polling direction (e.g. CURL_POLL_IN without CURL_POLL_OUT, or vice
versa), we previously had no way to disable the other direction's
notifier. Once created, a notifier stayed enabled and kept firing
curl_multi_socket_action() for events curl was no longer interested in.
Merge the read and write branches into a single helper that also
disables the notifier for a direction when its CURL_POLL_* flag is
absent from the mask. This measurably improves performance by avoiding
redundant curl_multi_socket_action() calls on sockets curl has asked
us to stop watching in a given direction.
Passing the browser command line and executable path to every WebContent
process just in case we load about:version always felt a bit weird. We
now use the WebUI framework to load this information on demand.
Previously, the LibWeb bindings generator would output multiple per
interface files like Prototype/Constructor/Namespace/GlobalMixin
depending on the contents of that IDL file.
This complicates the build system as it means that it does not know
what files will be generated without knowledge of the contents of that
IDL file.
Instead, for each IDL file only generate a single Bindings/<IDLFile>.h
and Bindings/<IDLFile>.cpp.
Route BroadcastChannel messages over IPC so matching channels can
receive them across WebContent and WebWorker processes, rather than only
within a single process.
Each channel now serializes its payload, sends it upward over IPC, and
receiving processes deliver it locally after matching by storage key and
channel name.
When we request the HTTP cookie for a SWR request, we were providing the
cookie to the standard request corresponding to the SWR request's ID.
This had two effects:
1. The SWR request would never finish.
2. If the corresponding standard request happened to be a connect-only
request, this would result in a crash as we were expecting it to have
gone through the normal fetch process.
This was seen on some articles on news.google.com.
The bytecode interpreter only needed the running execution context,
but still threaded a separate Interpreter object through both the C++
and asm entry points. Move that state and the bytecode execution
helpers onto VM instead, and teach the asm generator and slow paths to
use VM directly.
Generalize the backing store sharing abstraction into SharedImage, which
represents shared GPU memory independently of Skia and can be used to
share memory between different processes or different GPU contexts.
Split JS::ErrorData out of JS::Error so that it can be used both
by JS::Error and WebIDL::DOMException. This adds support for
Error.isError to DOMException, also letting us report DOMException
stack information to the console.
When a dedicated worker has an unhandled exception, we should propogate
that exception to be fired at the parent global. Fixes a timeout
in the included WPT test.
This prevents a race condition:
1. Try to connect a websocket
2. DNS lookup starts
3. JS causes the websocket to no longer be alive, and it is GCed
4. websocket_close() is called, but it doesn't find a websocket with
that websocket_id, so nothing happens
5. DNS lookup completes, and opens the websocket
6. This websocket never gets closed
By separately tracking which websockets we are trying to connect, we can
record the fact we tried to close it, and then the DNS lookup callback
can skip creating the now-unwanted websocket.
Replace the blocking spin_processing_tasks_with_source_until calls
in apply_the_history_step_after_unload_check() with an event-driven
ApplyHistoryStepState GC cell that tracks 5 phases, following the
same pattern used by CheckUnloadingCanceledState.
Key changes:
- Introduce ApplyHistoryStepState with phases:
WaitingForDocumentPopulation, ProcessingContinuations,
WaitingForChangeJobCompletion, WaitingForNonChangingJobs and Completed
- Add on_complete callbacks to apply_the_push_or_replace_history_step,
finalize_a_same_document_navigation,
finalize_a_cross_document_navigation, and
update_for_navigable_creation_or_destruction
- Remove spin_until from Document::open()
- Use null-document tasks for non-changing navigable updates and
document unload/destroy to avoid stuck tasks when documents become
non-fully-active
- Defer completely_finish_loading when document has no navigable yet,
and re-trigger post-load steps in activate_history_entry for documents
that completed loading before activation
Co-Authored-By: Shannon Booth <shannon@serenityos.org>
Move MachPortServer from LibWebView into LibIPC as MachBootstrapListener
and move the Mach message structs from MachMessageTypes.h into LibIPC.
These types are IPC infrastructure, not UI or platform concerns.
Consolidating them in LibIPC keeps the Mach bootstrap handshake
self-contained in a single library and removes LibWebView's dependency
on LibThreading.
Previously, the bootstrap handshake used a two-state machine
(WaitingForPorts / WaitingForReplyPort) to handle a race: the parent
registering transport ports and the child sending a bootstrap request
could arrive in either order, so whichever came first stored its half
and the second completed the handshake.
Eliminate the race by holding a mutex across spawn() and
register_child_transport(). Since the child cannot send a bootstrap
request before it exists, and the lock isn't released until its
transport is registered, handle_bootstrap_request() is guaranteed to
find the entry. This reduces the pending map to a simple pid-to-ports
lookup and collapses the two-variant state into two straightforward
branches: known child, or on-demand (non-child) caller like WebDriver.
Now that LibIPC uses Mach ports for transport on macOS, IOSurface port
rights can be sent as regular IPC message attachments instead of through
a separate ad-hoc Mach message side-channel. Introduce
Web::SharedBackingStore that wraps either a MachPort (macOS) or
ShareableBitmap (other platforms) with IPC encode/decode support,
unifying backing store allocation into the existing
did_allocate_backing_stores IPC message.
Registering multiple Mach port names with the bootstrap server at
runtime is not how macOS expects it to be used — the bootstrap server
is meant for static services, and the only reason we used it originally
was so child processes could reach back to the UI process.
Remove bootstrap_transport_over_socket(), which had both sides register
dynamic names with the bootstrap server and exchange them over a socket.
Instead, WebDriver and BrowserProcess connections now go through
MachPortServer instances directly. When a non-child process contacts a
MachPortServer, the server creates a port pair on demand (detected via
sysctl ppid check) and returns the local half immediately. This keeps
bootstrap server usage limited to the one original case: child processes
looking up their parent's MachPortServer.
WebDriver Session now runs its own MachPortServer per session.
--webdriver-content-path becomes --webdriver-mach-server-name on macOS.
Spare WebContent launches are skipped when a WebDriver session is active
to avoid bootstrap races.
On macOS, use Mach port messaging instead of Unix domain sockets for
all IPC transport. This makes the transport capable of carrying Mach
port rights as message attachments, which is a prerequisite for sending
IOSurface handles over the main IPC channel (currently sent via a
separate out-of-band path). It also avoids the need for the FD
acknowledgement protocol that TransportSocket requires, since Mach port
right transfers are atomic in the kernel.
Three connection establishment patterns:
- Spawned helper processes (WebContent, RequestServer, etc.) use the
existing MachPortServer: the child sends its task port with a reply
port, and the parent responds with a pre-created port pair.
- Socket-bootstrapped connections (WebDriver, BrowserProcess) exchange
Mach port names over the socket, then drop the socket.
- Pre-created pairs for IPC tests and in-message transport transfer.
Attachment on macOS now wraps a MachPort instead of a file descriptor,
converting between the two via fileport_makeport()/fileport_makefd().
The LibIPC socket transport tests are disabled on macOS since they are
socket-specific.
Instead of immediately firing fullscreenchange, defer that until
WebContent's client has confirmed that it is in fullscreen for the
content. The fullscreenchange is fired by the viewport change, so in
cases where the fullscreen transition is instantaneous (i.e. the
fullscreen state is entered at the exact moment the viewport expands),
the resize event should precede the fullscreenchange event, as the spec
requires.
This fixes the WPT element-request-fullscreen-timing.html test, which
was previously succeeding by accident because we were immediately
fullscreenchange upon requestFullscreen() being called, instead of
following spec and doing the viewport (window) resize in parallel. The
WPT test was actually initially intended to assert that the
fullscreenchange event follows the resize event, but the WPT runner
didn't actually have a different resolution for normal vs fullscreen
viewports, so the resize event doesn't actually fire in their setup. In
our headless mode, the default viewport is 800x600, and the fullscreen
viewport is 1920x1080, so we do fire a resize event when entering
fullscreen. Therefore, that imported test is reverted to assert that
the resize precedes the fullscreenchange.
We were conflating elements being the active element and elements being
activated. The :active pseudo class is supposed to be based on whether
an element will have its activation behavior run upon a button being
released.
Store whether an element is being activated as a flag that is set/reset
by EventHandler.
Doing this allows label elements to visually activate their control
without doing a weird paintable hack, so the Labelable classes have
been yeeted.
The RequestPipe uses a socketpair for streaming response body data
from RequestServer to WebContent. On macOS, the default socket buffer
size for AF_LOCAL sockets is only ~8KB, which meant every read/write
syscall could only transfer ~8KB at a time. For large responses, this
resulted in thousands of tiny reads with significant per-read overhead.
Increase the send and receive buffer sizes to 512KB, matching the
approach already used by IPC::TransportSocket. This dramatically
improves throughput for large response bodies -- for example, fetching
a 25MB file from localhost went from ~850ms to ~25ms in testing.
I'm unsure exactly why this is possible, but using a freed GC pointer
on macOS was causing a segmentation violation to reach our signal
handler, but instead of exiting, the process would get stuck.
To solve this, when terminal signals are received, just exit() instead
of trying to let it get to the default handler. This allows test-web to
get unstuck in cases like this, and instead of timing out and leaving a
zombie, count the test as a crash.
In particular, the issue was caused by calling top_level_traversable()
on a null Navigable. Since GC::Ptr's null checks are debug-only, it was
trying to access garbage m_parent fields. With or without the signal
handlers, this would result in an (unsurprising) EXC_BAD_ACCESS, as
observed by attaching lldb. Regardless of debuggers being attached, or
signal handlers being enabled, after the signal, the process would get
stuck and refuse to exit, even after a SIGKILL. Sampling the stuck
processes didn't seem to indicate that the program counter was moving
in this state, so I'm unsure what causes it to get stuck.
Previously, `create_paired()` returned two full Transport objects, and
callers would immediately call `from_transport()` on the remote side to
extract its underlying fd. This wasted resources: the remote
Transport's IO thread, wakeup pipes, and send queue were initialized
only to be torn down without ever sending or receiving a message.
Now `create_paired()` returns `{Transport, TransportHandle}` — the
remote side is born as a lightweight handle containing just the raw fd,
skipping all unnecessary initialization.
Also replace `release_underlying_transport_for_transfer()` (which
returned a raw int fd) with `release_for_transfer()` (which returns a
TransportHandle directly), hiding the socket implementation detail
from callers including MessagePort.
This fixes a race condition where a WebSocket would report a fatal
connection error instead of a clean close when the server dropped the
underlying connection immediately after sending a Close frame.
This fixes various timeouts in WPT, such as in:
https://wpt.live/websockets/Send-null.any.worker.html?wss
Instead of passing RequestServer and ImageDecoder socket FDs as
command-line arguments to WebWorker, send them over the main IPC channel
after launch. The worker-agent handoff now carries all three transport
handles (worker, RequestServer, ImageDecoder) so the connection path
matches WebContent.
Instead of passing RequestServer and ImageDecoder socket FDs as
command-line arguments to WebContent, send them over the main IPC
channel after launch. This unifies initial connection and reconnection
into a single code path.
Add IPC::TransportHandle as an abstraction for passing IPC
transports through .ipc messages. This replaces IPC::File at
all sites where a transport (not a generic file) is being
transferred between processes.
TransportHandle provides from_transport(),
clone_from_transport(), and create_transport() methods that
encapsulate the fd-to-socket-to-transport conversion in one
place. This is preparatory work for Mach port support on
macOS -- when that lands, only TransportHandle's internals
need to change while all .ipc definitions and call sites
remain untouched.
When a WebContent process receives a fatal signal, print a backtrace to
stderr before re-raising the signal. The backtrace is captured by
the test runner and written to a .stderr.html file.
Uses SA_RESETHAND so the handler runs only once, then resets to the
default disposition and re-raises the signal for normal crash behavior.
Consolidate the repeated socketpair + adopt + configure pattern from
4 call sites into a single Transport::create_paired() factory method.
This fixes inconsistent error handling and socket configuration across
call sites, and prepares for future mach port support on macOS.
Without IOSurface, the Metal rendering path introduced in #7956 hits a
VERIFY(iosurface_ref) failure and crashes on launch on Intel Macs.
The FIXME stated that the implementation of IOSurface does not work on
Intel macOS, but testing confirms it now works correctly.
Using a Promise in BackgroundAction was not doing anything since the
change to use a weak reference to the event loop, so let's just drop
that.
The thread will now always move itself (and therefore its callbacks)
over to the originating thread before completing, regardless of the
presence of callbacks. This ensures that ref counting remains on the
main thread.
In addition, BackgroundAction's completion callback can no longer
return errors. This functionality wasn't actually used anywhere, it was
a holdover from the behavior of Core::Promise.
WebView::FontPlugin was the only implementation of the abstract
FontPlugin base class. Its dependencies (LibGfx, LibCore) are
already visible to LibWeb.
Remove the virtual dispatch by making FontPlugin concrete and
absorbing the WebView::FontPlugin implementation directly.
EventLoopPluginSerenity was the only implementation of the abstract
EventLoopPlugin base class. Its methods simply wrapped Core::EventLoop
calls with GC function unwrapping.
Remove the virtual dispatch by making EventLoopPlugin concrete and
absorbing the EventLoopPluginSerenity implementation directly.
Request ids from a client connection must never be reused. This is also
enforced in the LibImageDecoderClient library by main thread asserts
around a monotonically increasing request id counter.
The Fetch spec defines HTTP whitespace as tab, LF, CR, and space.
Previously, trim_whitespace was also stripping vertical tab (U+000B)
and form feed (U+000C), which are not HTTP whitespace characters.
Switch to HTTP::normalize_header_value which matches the fetch
definition.
Fixes 4 subtests for WPT test:
https://wpt.live/cors/origin.htm
It's possible for the client (WebContent) to stop a request while the
request is waiting for an async callback to be invoked. So we cannot
assume the request itself will still be alive once these callbacks are
finally invoked.
We have a common pattern of creating a `WeakPtr<T>` from a reference and
passing that into a lambda, to then take the strong ref when the lambda
is executed. Add `weak_callback(Weakable, lambda)` that returns a lambda
that only invokes the callback if a strong ref exists, and passes it as
the first argument.
Stop converting between CSS and device pixels as part of rendering - the
display list should be as simple as possible, so convert to DevicePixels
once when constructing the display list.
Fixes the included imported test. Note that this required a minor
edit of the WPT import to work with our test harness setup to
try and create a non secure context setup as both file:// and
localhost are considered secure contexts.
These IPC methods should be expanded in the future to allow WebContent
to specify what UI elements should be kept/removed, for example, the
navigation UI.
The set_viewport_size and set_device_pixel_ratio IPC messages were sent
separately, potentially causing a race condition when the DPR changes
(e.g. moving a window between screens): the DPR message would arrive
and use a stale viewport size, computing a temporarily wrong CSS
viewport. Combine both into a single set_viewport IPC that updates the
device viewport size and DPR together.
Instead of passing through window's associated document's URL as
an extra argument to starting up a worker. This will allow for
improving the representation of 'outside settings' when setting
up a Worker.
This aligns our behaviour closer to other browsers, which
_mostly_ consider file scheme URLs as opaque. For test
purposes, allow overriding this behaviour with a commandline
flag.
We were effectively copying the logic from .set_viewport_size() into
PageClient, but we forgot to actually update the viewport size on device
pixel ratio changes.
This adds the --expose-experimental-interfaces command line flag to
enable experimental IDL interfaces. Any IDL interface with Experimental
in its exposed attributes will be disabled by default.
The problem is that by stubbing out or partially implementing interfaces
in LibWeb, we actually make some sites behave worse. For example, the
OffscreenCanvas interface being exposed makes sites believe we fully
support it, even though we don't. If the interface was not exposed,
these sites may fall back to ordinary canvas objects. Similarly, to
use YouTube, we currently have to patch out MSE interfaces.
This flag will allow developers to iteratively work on features,
without breaking such sites. We enable experimental interfaces during
tests.
We will need to propagate test mode behavior to both the WebContent and
WebWorker processes. By moving this handling to the UI process, we will
only need to update one location.
Especially for the disk cache, it always felt a bit sketchy to create
these objects on the global stack. We now create them in RequestServer's
main() where we can be more certain of destruction order, and pass them
to ConnectionFromClient instances.
This will allow the UI to request WebContent to properly close the top
level traversable when closing a tab. For example, this allows the site
to ask if the user is sure they want to leave, closes WebSocket
connections and more.
Add IPC messages and server-side implementation for streaming
animated image decode. Instead of decoding all frames upfront,
only decode an initial batch and keep the decoder alive for
on-demand frame requests.
New IPC messages:
- request_animation_frames: request decode of a batch of frames
- stop_animation_decode: clean up a decode session
- did_decode_animation_frames: deliver decoded frames to client
- did_fail_animation_decode: report decode errors
The existing did_decode_image message gains a session_id parameter
(0 for single-shot decode, non-zero for streaming sessions).
This adds a settings box to about:settings to allow users to limit the
disk cache size. This will override the default 5 GiB limit. We do not
automatically delete cache data if the new limit is suddenly less than
the used disk space; this will happen on the next request. This allows
multiple changes to the settings in a row without thrashing the cache.
In the future, we can add more toggles, such as disabling the disk
cache altogether.
