Produce JS-visible string results as UTF-16 at their source, including
numeric formatting, BigInt and BigFraction formatting, URI encoding,
console formatting, parser errors, regular expression errors, Intl and
Temporal records, LibUnicode locale boundaries, and LibWeb bindings.
Handle fractional radix formatting through the UTF-16 builder view.
Replace the generated C++ legacy codec implementations with a
small Rust wrapper around encoding_rs.
This keeps the existing LibTextCodec API while moving label lookup,
legacy decode/encode, validation, and streaming decoder state to Rust.
The generated index data and generator are no longer needed.
It also fixes several TextDecoder EOF cases due to a more correct
implementation. encoding_rs finalizes decoders according to the
Encoding Standard, so incomplete UTF-8/Big5 tails and malformed
UTF-16 surrogate tails produce the required single replacement at
end-of-queue instead of being dropped, buffered, or double-counted
by our old hand-written decoders.
Treat trailing UTF-8 prefixes with an invalid second byte as complete
input for streaming decode, so replacement characters are emitted in the
current chunk instead of being held until later input or finish. Keep
valid incomplete prefixes buffered across chunk boundaries.
Keep TextDecoderStream from holding continuation bytes after an invalid
lead byte at a chunk boundary. Add LibTextCodec and TextDecoderStream
coverage for invalid tails, valid split sequences, EOF partials,
surrogate sequences, and malformed continuation tails.
Previously we were inconsistent by generating code for enum definitions
but not generating code for dictionaries. With future changes to the
IDL generator to expose helpers to convert to and from IDL values
this produced circular depdendencies. To solve this problem, also
generate the dictionary definitions in bindings headers.
This is the streaming counterpart to TextDecoder, used by sites that
process responses with `pipeThrough(new TextDecoderStream())` or that
otherwise consume a Response body as decoded text.
The transform algorithm appends each chunk's bytes to the I/O queue
and decodes them via the underlying LibTextCodec decoder, holding back
any trailing partial UTF-8 sequence so chunk boundaries don't produce
spurious replacement characters. The flush algorithm emits a single
replacement character if a partial sequence was still pending. For
non-UTF-8 encodings the underlying decoders are stateless across
calls, so each chunk is decoded in full with nothing carried over.
