Replace the Rust URL form-encoding callback bridge into C++ TextCodec
with a direct encoding_rs encoder.
This keeps percent-encode-after-encoding entirely in Rust and removes
liburl_rust's dependency on LibTextCodec.
It also happens to fix ISO-2022-JP URL encoding of literal U+FFFD.
The LibTextCodec reverse lookup treats generated 0xFFFD table holes as
real JIS0208 mappings, so literal U+FFFD skipped the encoder-error
path. encoding_rs treats U+FFFD as unmappable, so URL encoding emits
the required numeric character reference.
Ever since PublicSuffixData was created, it was using "no star rule"
matching, which is what is needed for the address bar to distinguish
between a domain and a search. URL::Host on the other hand requires
the fallback star rule. Which rule is needed depends on the use case
of the PSL. Support both use cases by a flag in PublicSuffixData.
This adds a WebUI to view the local browsing history, with controls to
search and delete entries. The APIs used to search history are paginated
to prevent excessive query sizes.
Replace the generated public suffix table and custom matcher with a
direct LibURL PublicSuffixData implementation backed by libpsl. This
drops our PSL download/generator path and uses the same library already
used by libcurl.
Performance is comparable before and after, while LibURL binary size
is smaller.
PublicSuffixData handled trailing-dot hosts incorrectly when a PSL rule
matched, causing returned public suffix and registrable-domain results
to drop the trailing dot.
Make PublicSuffixData skip leading dots for matching, ignore a single
trailing dot while running the PSL algorithm, and append that trailing
dot back to returned results.
Move the registrable-domain helper from URL into PublicSuffixData and
name it find_matching_registrable_domain().
This keeps it alongside find_matching_public_suffix(), making it clear
that both APIs only return results matched from the PSL data, while
Host::public_suffix() implements the URL Standard fallback to the
top-level domain.
Rename PublicSuffixData's raw lookup helpers to make it clear that they
only return public suffixes matched from the PSL data.
This distinguishes them from Host::public_suffix(), which implements the
URL Standard definition and falls back to the top-level domain when no
PSL rule matches.
We need both layers because address bar handling needs the raw lookup to
decide whether input should be treated as a URL or as a search.
After batch-processing the userinfo, the authority state advanced the
parse pointer onto the '@' and relied on the end-of-loop increment to
step past it. That increment uses the byte length of the authority's
first code point, not the delimiter's. When the first code point was
multi-byte, the pointer overshot into the middle of a code point and
the next iteration sliced the string at a non-char boundary causing the
process to panic. We now step past the delimiter explicitly so we don't
need to rely on the end of loop increment to do so.
The Rust URL FFI used String::from_utf8_lossy() on raw bytes from C++.
That was not equivalent to the old C++ parser behavior for WTF-8 encoded
surrogates. For example a byte sequence such as ED A0 80 was treated as
three invalid UTF-8 bytes, producing three U+FFFD replacement
characters.
Instead use String::from_utf8_with_replacement_character and let Rust
borrow the input as &str.
Copy contiguous ASCII path/query runs in one slice append, instead of
processing each code point individually. Fall back to the normal state
machine for delimiters, percent escapes, non-ASCII input, and validation
cases.
This gives a 2.4x speedup on a benchmark parsing common web URLs.
Passing the browser command line and executable path to every WebContent
process just in case we load about:version always felt a bit weird. We
now use the WebUI framework to load this information on demand.
This page renders the bookmarks as a tree and hook context menu events
up to the UI's bookmarks bar context menus to allow editing bookmarks.
Users can also drag-and-drop bookmark items around.
Let's not have to know off-hand that we need to update Environments.cpp
when adding a new WebUI. It's more obvious just below where we define
the URLs.
Add `ECMAScriptRegex`, LibRegex's C++ facade for ECMAScript regexes.
The facade owns compilation, execution, captures, named groups, and
error translation for the Rust backend, which lets callers stop
depending on the legacy parser and matcher types directly. Use it in the
remaining non-LibJS callers: URLPattern, HTML input pattern handling,
and the places in LibHTTP that only needed token validation.
Where a full regex engine was unnecessary, replace those call sites with
direct character checks. Also update focused LibURL, LibHTTP, and WPT
coverage for the migrated callers and corrected surrogate handling.
execResult may contain additional captures from nested groups
in user-provided regexp parts, exceeding the number of
URLPattern groups.
Fixes a crash in the updated WPT test.
See: https://github.com/whatwg/urlpattern/commit/203d435c32
There have been a few times I have wanted to add a debug log in
one of these functions, but currently that causes a massive rebuild.
Let's just move these out of line.
This aligns our behaviour closer to other browsers, which
_mostly_ consider file scheme URLs as opaque. For test
purposes, allow overriding this behaviour with a commandline
flag.
This was previously a depdency due to the use of
Crypto::get_secure_random for the nonce used for an opaque origin.
Now that this has been moved to AK, we no longer have any dependency
on LibCrypto.
Previously the authority state was parsed character-by-character in a
loop, appending each character to a buffer. When an '@' character was
encountered, the entire buffer would be re-processed to extract and
percent-encode the username and password portions.
This created O(n^2) behavior for URLs with multiple '@' characters, as
each '@' would trigger reprocessing of all previously buffered content.
This commit changes the authority state parser to process the authority
section in chunks rather than character-by-character:
1. Find the next delimiter ('@', '/', '?', '#', or '\' for special URLs)
2. Process the entire chunk up to that delimiter at once
3. Directly extract username/password from the chunk without buffering
With an additional change of switching to:
iterator_at_byte_offset_without_validation (which does not have any
loops), this reduces the time complexity to O(n) and the included
test found by fuzzing to actually complete parsing :^)
LibURL previously assigned a default port to the IRC schemes,
a carryover from SerenityOS where IRC is supported.
This behavior deviates from the URL Standard and affects URL parsing by
eliding an explicitly specified port when it matches the default (this
is considered a legacy behaviour of the web URL schemes). Remove the IRC
default port to restore spec-compliant behavior.
AK/Random is already the same as SecureRandom. See PR for more details.
ProcessPrng is used on Windows for compatibility w/ sandboxing measures
See e.g. https://crbug.com/40277768
The same-origin domain check always returned true if only the scheme
matched. This was because of missing steps to check for the origin's
domain, which didn't exist. Add this concept to URL::Origin, even
though we do not use it at this stage in document.domain setter.
Co-Authored-By: Luke Wilde <luke@ladybird.org>
Instead of '*', which is a nonsensical value. Similar to what we do
for determining the public suffix, if no match could be made via the
PSL algorithm, then take everything after the second dot as
the registrable domain.
This prevents us from considering e.g. b.b.example and
a.example as the same site.
This required some changes in LibURL & LibIPC since it has its own
definition of an BlobURLEntry. For now, we don't have a concrete usage
of MediaSource in LibURL so it is defined as an empty struct.
This removes one FIXME in an idl file.
Our floating point number parser was based on the fast_float library:
https://github.com/fastfloat/fast_float
However, our implementation only supports 8-bit characters. To support
UTF-16, we will need to be able to convert char16_t-based strings to
numbers as well. This works out-of-the-box with fast_float.
We can also use fast_float for integer parsing.
For the AO defined in the URL specification, in the case the
domain does not match against the PSL, we should be returning
the TLD. This fixes a crash for a bunch of WPT tests using the
Document.domain setter when the test is being served by WPT
locally.
We should be doing similar logic in registrable_domain, but that
unfortunately runs into some other issues, so just leave a FIXME
for now.
