When there is an active insertion point, it's necessary to tokenize
code-point-by-code-point to handle the case of document.write being
used to insert a named character reference one code point at a time.
However, when there is no insertion point defined, looking ahead at the
input and doing the matching all-at-once is more efficient since it
allows:
- Avoiding the work done in next_code_point between each code point
being matched (leading to better CPU cache usage in theory)
- Skipping ahead to the end of the match all at once, which does less
work overall than the equivalent number of next_code_point calls
(that is, skip(N) does less work than next_code_point called N times)
In my benchmarking, this provides a small performance boost (fewer
instructions, fewer cpu cycles, fewer branch misses) essentially for
free.
7096a2892e