355fb6b825
Avoid building a temporary Rust token vector before calling back into C++. The tokenizer now invokes the callback as each token is produced, while borrowing the already-filtered input for source slices. Reserve an initial C++ token capacity from the input size so the common path avoids repeated growth while appending the converted tokens. With this change, the Rust CSS tokenizer is now ~1.3x faster than the C++ CSS tokenizer at churning through all the https://vercel.com/ CSS. |
||
|---|---|---|
| .. | ||
| src | ||
| build.rs | ||
| Cargo.toml | ||
| cbindgen.toml | ||