Produce JS-visible string results as UTF-16 at their source, including
numeric formatting, BigInt and BigFraction formatting, URI encoding,
console formatting, parser errors, regular expression errors, Intl and
Temporal records, LibUnicode locale boundaries, and LibWeb bindings.
Handle fractional radix formatting through the UTF-16 builder view.
Move the remaining LibJS primitive string users to UTF-16 views and
strings. Remove the primitive string UTF-8 accessors and byte-string
coercion paths so new callers cannot rely on the old storage model.
Replace the remaining UTF-16-mode StringBuilder instances in LibWeb
with Utf16StringBuilder when they assemble Utf16String values or UTF-16
text views. This keeps those paths in UTF-16 throughout and uses
explicit ASCII append helpers for ASCII-only pieces.
Keep StringBuilder in place for JSON, markup, CSS serialization, and
other byte-oriented string construction paths.
Use String::from_utf8_with_replacement_character() directly for Infra's
parse JSON bytes algorithm instead of constructing a more generic
TextCodec UTF-8 decoder.
The spec declares these as a byte sequence, which we then implemented as
a ByteBuffer. This has become pretty awkward to deal with, as evidenced
by the plethora of `MUST(ByteBuffer::copy(...))` and `.bytes()` calls
everywhere inside Fetch. We would then treat the bytes as a string
anyways by wrapping them in StringView everywhere.
We now store these as a ByteString. This is more comfortable to deal
with, and we no longer need to continually copy underlying storage (as
ByteString is ref-counted).
This work is largely preparatory for an upcoming HTTP header refactor.
This has quite a lot of fall out. But the majority of it is just type or
UDL substitution, where the changes just fall through to other function
calls.
By changing property key storage to UTF-16, the main affected areas are:
* NativeFunction names must now be UTF-16
* Bytecode identifiers must now be UTF-16
* Module/binding names must now be UTF-16
This porting effort makes it pretty clear we will want a UTF-16-aware
GenericLexer. But for now, we can actually make ASCII assumptions about
what we are parsing, and act accordingly.
Which has an optmization if both size of the string being passed
through are FlyStrings, which actually ends up being the case
in some places during selector matching comparing attribute names.
Instead of maintaining more overloads of
Infra::is_ascii_case_insensitive_match, switch
everything over to equals_ignoring_ascii_case instead.
isomorphic encoding a value that has already been encoded will
result in garbage data. `response_headers` is already encoded in
ISO-8859-1/latin1, we cannot use `from_string_pair`, as it triggers
ISO-8859-1/latin1 encoding.
Follow-up of https://github.com/LadybirdBrowser/ladybird/pull/1893
