Commit graph

9 commits

Author SHA1 Message Date
Andreas Kling
b81269e78b Libraries: Clean up UTF-16 source text paths
Store parser errors, source range filenames, source code filenames,
module source, and Rust parser errors as UTF-16 where they flow back
into JavaScript-visible strings. Keep byte-oriented source buffers
byte-backed.

Remove temporary PrimitiveString, ByteString, and UTF-8 detours from
JSON, RegExp, module debug logging, print formatting, and tests.
2026-06-22 19:51:25 +02:00
Andreas Kling
b6bef6b688 Libraries: Use UTF-16 for JS-visible runtime strings
Produce JS-visible string results as UTF-16 at their source, including
numeric formatting, BigInt and BigFraction formatting, URI encoding,
console formatting, parser errors, regular expression errors, Intl and
Temporal records, LibUnicode locale boundaries, and LibWeb bindings.

Handle fractional radix formatting through the UTF-16 builder view.
2026-06-22 19:51:25 +02:00
Andreas Kling
7025dd1fa7 Libraries: Parse JS strings from UTF-16
Thread UTF-16 string input through JSON, script parsing, Date parsing,
Intl option parsing, Temporal parsing, and the helper library boundaries
that feed those parsers. Preserve ASCII fast paths where the source data
is known to be ASCII.
2026-06-22 19:51:25 +02:00
Timothy Flynn
e6c68e9e91 LibUnicode: Remove unused UTF-32 segmenter support 2026-06-06 18:42:18 +02:00
Tim Ledbetter
f161215f53 LibUnicode: Add an ASCII fast path for line break segmentation
Add a Segmenter implementation that implements the UAX#14 line breaking
rules applicable to ASCII text. This avoids the need to build an ICU
BreakIterator for the majority of text on the web.
2026-05-05 18:59:07 +02:00
Tim Ledbetter
ae0f5ef9ce LibUnicode+LibWeb: Add infrastructure for line segmentation using ICU
No behavior change This is needed for correct UAX#14 line breaking.
2026-02-14 16:23:18 -05:00
Andreas Kling
8cdfbfed49 LibUnicode+LibWeb: Add fast path grapheme segmenter for ASCII text
For ASCII text, every character is its own grapheme - there are no
combining characters or emoji sequences. This means grapheme boundary
detection is trivial: next_boundary(i) is simply i+1.

This commit adds AsciiGraphemeSegmenter, a simple Segmenter subclass
that performs O(1) boundary lookups without any ICU overhead.

TextNode::grapheme_segmenter() now checks if the text is ASCII and uses
this fast path, avoiding expensive ICU BreakIterator cloning and
boundary detection for the common case of ASCII-only text.
2026-01-11 11:10:19 +01:00
Timothy Flynn
8b6e3cb735 LibWeb+LibUnicode+WebContent: Port DOM:CharacterData to UTF-16
This replaces the underlying storage of CharacterData with Utf16String
and deals with the fallout.
2025-07-24 19:00:20 +02:00
Timothy Flynn
93712b24bf Everywhere: Hoist the Libraries folder to the top-level 2024-11-10 12:50:45 +01:00
Renamed from Userland/Libraries/LibUnicode/Segmenter.h (Browse further)