Store reftest expectations as a list of match/mismatch reference URLs
and evaluate each reference in sequence.
This lets test-web run WPT reftests that specify multiple reference
relations, such as tests with both match and mismatch links.
Teach test-web to expose the UI-process history dump. Add focused
navigation tests for same-document traversal, fallback traversal, and
cross-document browser back and forward behavior. The expectations
assert document state and the UI-owned history snapshot.
Scan sources of Text tests during collection to identify test variants,
using a very basic HTML parser. All tests with variants use the same
pattern for their meta tags, so even this is overkill for what we need,
but it should work for any future tests that happen to be arranged a
little differently.
Load tests that require the echo server through a per-test
*.localhost hostname instead of plain localhost. This prevents shared
per-host browser state, from leaking between parallel or repeated test
runs.
Update HTTPTestServer to create echo URLs on the current origin when a
test is already loaded from the echo server, preserving same-origin
behavior for tests that inspect echo-served iframes.
Problem: Text-mode tests that perform a top-level cross-document
navigation can hang forever — even when the target page reports PASS.
For example, the connect-src test in Tests/LibWeb/Text/input/navigation/
has been intermittently timing out on CI for months, and has its own
in-test workaround. But that’s a mitigation that just narrows the race.
Cause: The Text-mode on_load_finish handler in test-web/main.cpp filters
by URL and bails out if the loaded URL doesn’t match the test’s original
URL — to ignore subframe loads. But page_did_finish_loading is already
gated on navigable->is_traversable(). So, it’s top-level-only — and the
URL filter is actually rejecting load events from top-level navigations
away from the original test URL.
When the original page’s did_finish_loading IPC hasn’t reached test-web
by the time the navigation target loads and signals PASS, the target’s
did_finish_loading is the only surviving load notification — and the URL
filter masks it. So, did_finish_loading is never set test-side. And so
the test-finish handler then bails on the did_finish_loading &&
did_check_variants check — and the test hangs.
Fix: Drop the URL filter from the Text-mode on_load_finish handler. Any
top-level load (the original page, or any subsequent navigation) now
sets did_finish_loading. The variant-loading logic is unchanged: The
second variant-check just no-ops — because test.variant is already set
from the first response.
Look for screenshot expectations in an expected-<platform>
directory before using the shared expected directory. This lets tests
that cover system-derived rendering keep platform-specific baselines
without making other platforms use the same PNG.
Add internals helpers so text tests can install local blocker rules and
toggle content blocking without browser-process IPC. Clear installed
rules after each non-crashed test-web case so blocker state cannot leak
into the next test.
Rename the local content blocking implementation and its tests from
ContentFilter to ContentBlocker while keeping the existing substring
matcher backend and behavior.
Update the WebContent IPC method, WebView option names, debug toggle,
and default config file name to use content blocker terminology.
Keep a small harness-status.txt file in the test-web results
directory and refresh it whenever tests start, finish, or the run stops
early. This leaves the last known active views, counts, and remaining
test count behind even if the harness does not reach its normal result
generation path.
Also report helper-process log write failures instead of treating them
as "no logs". That makes artifact gaps visible instead of silently
losing the only clue from a helper process failure.
Log active views and tests when test-web stops before all tests have
completed. Reuse the same reporting for signal handling, and accept any
future signal registration without asserting while explaining shutdown.
Also log observed helper process exits with decoded Unix wait status so
CI logs show whether a process exited normally or died from a signal.
Record the scroll geometry that CompositorThread can safely reason about
while the main thread is painting. The snapshot contains stable scroll
node IDs, parent links, scroll bounds, sticky inputs, and wheel blocker
regions in display-list coordinate space.
Add AsyncScrollingState and AsyncScrollTree under LibWeb/Compositor. The
state is the immutable main-thread snapshot; the tree is the mutable
compositor-side copy that can replay scroll deltas and sticky offsets
without touching DOM, layout, or paintables.
Expose the state through internals and add text tests for tree shape,
parent links, sticky areas, blocker hit testing, nested scrollers, and
admission decisions. Keep the directory skipped unless the feature is
enabled with --enable-async-scrolling.
Previously, we signalled completion of a PDF test once the
`pagesloaded` event fired. This fires once page metadata is initialized
but the text and annotation layers are rendered asynchronously after
that. We now wait for the text and annotation layers of the first page
to render. The test runner will currently fail for PDFs with more than
one page bceause pages are loaded lazily once they are visible and
there's no guarantee that anything beyond the first page will be loaded.
This script waits until fonts have loaded and waits for 2 animation
frames before signalling test completion. This is the same mechanism
already used for ref and crash tests.
Deferred warnings were originally intended to suppress output during
live display, ostensibly to avoid glitch scrolls.
Then, 1af74d1a7c added log capture to the
test-web process. Suddenly, deferred warnings became deadly because
they're flushed during a tiny window after the capture notifier has
stopped draining the tee pipe but before stderr is restored.
This caused a deadlock at exit. The fix is to remove this system and
call warnln directly because display integrity is now protected by
other means.
This prevents glitches where stale test status lines can bleed above
the live display list. Log this additional captured output so it can
be reviewed if a main or helper process failure is suspected.
Use files instead of buffers. Consolidate stderr and stdout into one
view. Break handling out into TestRunCapture and CaptureFile helper
classes. We will use them to log output from ALL processes in the next
commit.
When test-web runs inside Claude Code or Codex, per-test progress lines
are suppressed. These environments are non-interactive and the per-test
output (potentially thousands of lines) wastes the agent's context
window without adding value. The agent only needs the pass/fail summary
to decide next steps.
Quiet mode preserves: the "Running N tests..." line, the summary table,
non-passing test names, and the results URL.
This is a little more complicated than screenshot tests, because we deal
with two separate page loads, for the test and expectation pages. When
loading the expectation page, we clear the did_inject_js flag and also
store the URL so that we can compare that later.
Keep track of whether we've injected the JS, and avoid doing so again.
Also skip this entirely unless we're actually dealing with the test's
URL and not a subframe. This check is already done on dump tests.
The MutationObserver was not disconnected after firing, so any other
modifications to the document's class attribute after that point would
signal the test completing again. This is unlikely to happen in
practice as we shouldn't be running any JS after completing a test, but
it's harmless to disconnect it.
We had three separate timeout timer sites in `test-web`, which are now
deduplicated into a single one. This single timeout timer is started
_before_ we start loading `about:blank`, which potentially catches more
timeouts when a WebContent process is left in a broken state by a
previous test.
Previously, if you had a mistake in a test path, you'd get an unhelpful
error like this when running test-web:
> Runtime error: realpath: No such file or directory (errno=2)
This is quite confusing if other changes have been made, as the error
could have happened anywhere.
With this change, the same mistake now is first reported as:
> Unable to get real path for TestConfig.ini entry
> '/home/sam/ladybird/Tests/LibWeb/FLORB.html': realpath: No such file
> or directory (errno=2)
Then followed by the original error reported by LibMain. This makes it
immediately obvious what the issue is, and hopefully will save anyone
else going on a wild goose chase trying to find a build issue that isn't
there. 😅
When a path in TestConfig.ini ends in a /, treat it as a directory and
recursively enumerate all tests within it to the requisite collection
of paths.
We can immediately use this to drop a bunch of lines from the config
that specify individual IndexedDB tests should use the HTTP server.
If provided, test-web batches & results will be partitioned in
multiple runs. Each run will execute the same set of tests, and
non-pass results will be stored in run1, run2, run6, etc directories.
Previously, --rebaseline would unconditionally overwrite expected PNGs
before comparing, causing every screenshot test expectation to be
regenerated even when the actual screenshot already matched. Restructure
to load and compare first, only writing the new expectation on mismatch
or when the expected file doesn't exist yet.
Instead of rendering a reference HTML page that wraps an <img> tag
pointing to a PNG, Screenshot tests now load the expected PNG directly
from disk and compare it against the rendered screenshot. This
eliminates the indirection of loading and rendering a second page just
to display a static image.
This also means --rebaseline now works for Screenshot tests, generating
the expected PNG automatically instead of requiring manual screenshot
capture and placement.
Changes:
- Add TestMode::Screenshot with its own collector and runner
- Move PNGs from Screenshot/images/ to Screenshot/expected/ with
normalized names matching input filenames
- Remove all 92 reference HTML wrapper files and the images/
directory
- Remove <link rel="match"> from all 94 Screenshot input HTML
files
- Update add_libweb_test.py Screenshot boilerplate accordingly
- Add Screenshot mode to results viewer image comparison tabs
