Ever since PublicSuffixData was created, it was using "no star rule"
matching, which is what is needed for the address bar to distinguish
between a domain and a search. URL::Host on the other hand requires
the fallback star rule. Which rule is needed depends on the use case
of the PSL. Support both use cases by a flag in PublicSuffixData.
Replace the generated public suffix table and custom matcher with a
direct LibURL PublicSuffixData implementation backed by libpsl. This
drops our PSL download/generator path and uses the same library already
used by libcurl.
Performance is comparable before and after, while LibURL binary size
is smaller.
PublicSuffixData handled trailing-dot hosts incorrectly when a PSL rule
matched, causing returned public suffix and registrable-domain results
to drop the trailing dot.
Make PublicSuffixData skip leading dots for matching, ignore a single
trailing dot while running the PSL algorithm, and append that trailing
dot back to returned results.
Move the registrable-domain helper from URL into PublicSuffixData and
name it find_matching_registrable_domain().
This keeps it alongside find_matching_public_suffix(), making it clear
that both APIs only return results matched from the PSL data, while
Host::public_suffix() implements the URL Standard fallback to the
top-level domain.
Rename PublicSuffixData's raw lookup helpers to make it clear that they
only return public suffixes matched from the PSL data.
This distinguishes them from Host::public_suffix(), which implements the
URL Standard definition and falls back to the top-level domain when no
PSL rule matches.
We need both layers because address bar handling needs the raw lookup to
decide whether input should be treated as a URL or as a search.
Instead of '*', which is a nonsensical value. Similar to what we do
for determining the public suffix, if no match could be made via the
PSL algorithm, then take everything after the second dot as
the registrable domain.
This prevents us from considering e.g. b.b.example and
a.example as the same site.
For the AO defined in the URL specification, in the case the
domain does not match against the PSL, we should be returning
the TLD. This fixes a crash for a bunch of WPT tests using the
Document.domain setter when the test is being served by WPT
locally.
We should be doing similar logic in registrable_domain, but that
unfortunately runs into some other issues, so just leave a FIXME
for now.
It is confusing to have both URL::Host::public_suffix and
URL:get_public_suffix, both with slightly different semantics.
Instead, use PublicSuffixData for cases that just want a direct
match against the list, and URL::Host::public_suffix in LibWeb
land as the URL spec defined AO.
We were using the public suffix of the URL's host as its registrable
domain. But the registrable domain is actually the public suffix plus
one additional label.
Specifically, after implementing some recent spec changes to navigables,
we end up calling `get_public_suffix("localhost")` here, which returns
OptionalNone. This would previously crash.
Our get_public_suffix() seems a little incorrect. From the spec:
> If no rules match, the prevailing rule is "*".
> https://github.com/publicsuffix/list/wiki/Format#algorithm
However, ours returns an empty Optional in that case. To avoid breaking
other users of it, this patch modifies Host's uses of it, rather than
the function itself.
This lets us move a few Host-related functions (like serialization and
checks for what the Host is) into Host instead of having them dotted
around the codebase.
For now, the interface is still very Variant-like, to avoid having to
change quite so much in one go.
