54a172f3b5
Add a dump-html-tokens utility that prints HTML tokenizer output in a stable text format, along with focused tokenizer input and expectation files. Include silent and repeated-iteration modes so the utility can also be used for local tokenizer throughput checks without changing the dumped token format.
12 lines
271 B
Text
12 lines
271 B
Text
Character U+0009 @0:4
|
|
Character U+000A @0:9
|
|
Character U+000D @0:14
|
|
Character U+0020 ' ' @0:19
|
|
Character U+007F @0:25
|
|
Character U+20AC @0:31
|
|
Character U+0178 @0:37
|
|
Character U+FFFD @0:41
|
|
Character U+10FFFF @0:51
|
|
Character U+FFFD @0:61
|
|
Character U+000A @1:0
|
|
EndOfFile @1:0
|