dmaax/ladybird
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ladybird/Libraries/LibWeb/HTML/Parser/HTMLParser.h

243 lines
8.8 KiB
C
Raw Normal View History

LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
/*
Meta: Update my e-mail address everywhere
2024-10-04 08:19:50 -03:00
* Copyright (c) 2020-2022, Andreas Kling <andreas@ladybird.org>
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
*
Everything: Move to SPDX license identifiers in all files. SPDX License Identifiers are a more compact / standardized way of representing file license information. See: https://spdx.dev/resources/use/#identifiers This was done with the `ambr` search and replace tool. ambr --no-parent-ignore --key-from-file --rep-from-file key.txt rep.txt *
2021-04-22 05:24:48 -03:00
* SPDX-License-Identifier: BSD-2-Clause
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
*/
#pragma once
LibWeb: Add Web::HTML::parse_legacy_color_value This function follows the "rules for parsing a legacy color value" which is used in some legacy attributes, such as 'bgcolor' in the body element.
2023-05-28 00:04:40 -03:00
#include <LibGfx/Color.h>
LibWeb: Make the HTMLParser GC-allocated This prevents a reference cycle between a HTMLParser opened via document.open() and the document. It was one of many things keeping some documents alive indefinitely.
2022-10-17 05:46:11 -03:00
#include <LibJS/Heap/Cell.h>
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
#include <LibWeb/DOM/Node.h>
LibWeb: Move the HTML parser into HTML/Parser/
2020-07-28 14:18:23 -03:00
#include <LibWeb/HTML/Parser/HTMLTokenizer.h>
#include <LibWeb/HTML/Parser/ListOfActiveFormattingElements.h>
#include <LibWeb/HTML/Parser/StackOfOpenElements.h>
LibWeb: Use Content-Type header to set document encoding Co-authored-by: Shannon Booth <shannon@serenityos.org>
2024-10-20 05:39:50 -03:00
#include <LibWeb/MimeSniff/MimeType.h>
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
LibWeb: Store a SpeculativeHTMLParser on the HTML Parser The parser was previously added, but unused. Actually attaching one to the HTML Parser will let us test the limits of Swift interop.
2025-04-04 12:20:27 -03:00
#ifdef LIBWEB_USE_SWIFT
# include <LibGC/ForeignCell.h>
namespace Web {
Meta: Enforce newlines around namespaces This has come up several times during code review, so let's just enforce it using a new clang-format 20 option.
2025-05-13 08:06:33 -03:00
LibWeb: Store a SpeculativeHTMLParser on the HTML Parser The parser was previously added, but unused. Actually attaching one to the HTML Parser will let us test the limits of Swift interop.
2025-04-04 12:20:27 -03:00
class SpeculativeHTMLParser;
Meta: Enforce newlines around namespaces This has come up several times during code review, so let's just enforce it using a new clang-format 20 option.
2025-05-13 08:06:33 -03:00
LibWeb: Store a SpeculativeHTMLParser on the HTML Parser The parser was previously added, but unused. Actually attaching one to the HTML Parser will let us test the limits of Swift interop.
2025-04-04 12:20:27 -03:00
}
#endif
LibWeb: Move HTML classes into the Web::HTML namespace
2020-07-28 13:20:36 -03:00
namespace Web::HTML {
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
#define ENUMERATE_INSERTION_MODES \
__ENUMERATE_INSERTION_MODE(Initial) \
__ENUMERATE_INSERTION_MODE(BeforeHTML) \
__ENUMERATE_INSERTION_MODE(BeforeHead) \
__ENUMERATE_INSERTION_MODE(InHead) \
__ENUMERATE_INSERTION_MODE(InHeadNoscript) \
__ENUMERATE_INSERTION_MODE(AfterHead) \
__ENUMERATE_INSERTION_MODE(InBody) \
__ENUMERATE_INSERTION_MODE(Text) \
__ENUMERATE_INSERTION_MODE(InTable) \
__ENUMERATE_INSERTION_MODE(InTableText) \
__ENUMERATE_INSERTION_MODE(InCaption) \
__ENUMERATE_INSERTION_MODE(InColumnGroup) \
__ENUMERATE_INSERTION_MODE(InTableBody) \
__ENUMERATE_INSERTION_MODE(InRow) \
__ENUMERATE_INSERTION_MODE(InCell) \
__ENUMERATE_INSERTION_MODE(InSelect) \
__ENUMERATE_INSERTION_MODE(InSelectInTable) \
__ENUMERATE_INSERTION_MODE(InTemplate) \
__ENUMERATE_INSERTION_MODE(AfterBody) \
__ENUMERATE_INSERTION_MODE(InFrameset) \
__ENUMERATE_INSERTION_MODE(AfterFrameset) \
__ENUMERATE_INSERTION_MODE(AfterAfterBody) \
__ENUMERATE_INSERTION_MODE(AfterAfterFrameset)
LibWeb: Make the HTMLParser GC-allocated This prevents a reference cycle between a HTMLParser opened via document.open() and the document. It was one of many things keeping some documents alive indefinitely.
2022-10-17 05:46:11 -03:00
class HTMLParser final : public JS::Cell {
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
GC_CELL(HTMLParser, JS::Cell);
GC_DECLARE_ALLOCATOR(HTMLParser);
LibWeb: Make the HTMLParser GC-allocated This prevents a reference cycle between a HTMLParser opened via document.open() and the document. It was one of many things keeping some documents alive indefinitely.
2022-10-17 05:46:11 -03:00
LibWeb: Implement state switch for "[CDATA[" in HTML parser
2022-02-15 15:52:45 -03:00
friend class HTMLTokenizer;
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
public:
LibWeb: Rename HTMLDocumentParser => HTMLParser
2021-09-25 18:15:48 -03:00
~HTMLParser();
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
static GC::Ref<HTMLParser> create_for_scripting(DOM::Document&);
static GC::Ref<HTMLParser> create_with_uncertain_encoding(DOM::Document&, ByteBuffer const& input, Optional<MimeSniff::MimeType> maybe_mime_type = {});
static GC::Ref<HTMLParser> create(DOM::Document&, StringView input, StringView encoding);
LibWeb: Implement encoding sniffing algorithm This patch implements the HTML specification's "encoding sniffing algorithm", which is used when no encoding can be obtained from the Content-Type header (either because it doesn't contain a charset=...) value or the file has not been opened via HTTP (as with local files). It also modifies the creator of the HTMLDocumentParser to use the new HTMLDocumentParser::create_with_uncertain_encoding static method, which runs the encoding sniffing algorithm before instantiating the parser. This now allows us to load local HTML pages (or remote pages without a charset specified in the 'Content-Type' header) with a non-UTF-8 encoding such as 'windows-1252'. This would previously crash the browser. :^)
2021-05-12 05:47:12 -03:00
LibWeb: Stop parsing after `document.write` at the insertion point If a call to `document.write` inserts an incomplete HTML tag, e.g.: document.write("<p"); we would previously continue parsing the document until we reached a closing angle bracket. However, the spec states we should stop once we reach the new insertion point.
2024-02-18 14:45:53 -03:00
void run(HTMLTokenizer::StopAtInsertionPoint = HTMLTokenizer::StopAtInsertionPoint::No);
AK+LibURL: Move AK::URL into a new URL library This URL library ends up being a relatively fundamental base library of the system, as LibCore depends on LibURL. This change has two main benefits: * Moving AK back more towards being an agnostic library that can be used between the kernel and userspace. URL has never really fit that description - and is not used in the kernel. * URL _should_ depend on LibUnicode, as it needs punnycode support. However, it's not really possible to do this inside of AK as it can't depend on any external library. This change brings us a little closer to being able to do that, but unfortunately we aren't there quite yet, as the code generators depend on LibCore.
2024-03-18 00:22:27 -03:00
void run(const URL::URL&, HTMLTokenizer::StopAtInsertionPoint = HTMLTokenizer::StopAtInsertionPoint::No);
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
static void the_end(GC::Ref<DOM::Document>, GC::Ptr<HTMLParser> = nullptr);
LibWeb: Make HTMLParser::the_end() callable from outside This is a little awkward: The spec requires when loading media documents or ones that don't have a DOM, that we "act as if the user agent had stopped parsing document" which means following this algorithm. Only a few steps require an HTMLParser, but those that do, involve reaching into its internals. The simplest solution I could think of (other than duplicating this fairly hefty function) is making it static and taking a Document and optional HTMLParser as parameters.
2023-12-19 09:51:34 -03:00
LibWeb: Move DOM classes into the Web::DOM namespace LibWeb keeps growing and the Web namespace is filling up fast. Let's put DOM stuff into Web::DOM, just like we already started doing with SVG stuff in Web::SVG.
2020-07-26 14:37:56 -03:00
DOM::Document& document();
LibWeb: Implement unsafe HTML parsing methods Both Element's and ShadowRoot's setHTMLUnsafe, and Document's static parseHTMLUnsafe methods are implemented.
2024-06-25 16:55:58 -03:00
enum class AllowDeclarativeShadowRoots {
No,
Yes,
};
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
static Vector<GC::Root<DOM::Node>> parse_html_fragment(DOM::Element& context_element, StringView, AllowDeclarativeShadowRoots = AllowDeclarativeShadowRoots::No);
LibWeb: Update HTML fragment serialization for declarative shadow DOM
2024-06-25 05:49:54 -03:00
enum class SerializableShadowRoots {
No,
Yes,
};
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
static String serialize_html_fragment(DOM::Node const&, SerializableShadowRoots, Vector<GC::Root<DOM::ShadowRoot>> const&, DOM::FragmentSerializationMode = DOM::FragmentSerializationMode::Inner);
LibWeb: Implement fragment parsing and use it for Element.innerHTML This patch implements most of the HTML fragment parsing algorithm and ports Element::set_inner_html() to it. This was the last remaining user of the old HTML parser. :^)
2020-06-25 18:42:08 -03:00
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
enum class InsertionMode {
#define __ENUMERATE_INSERTION_MODE(mode) mode,
ENUMERATE_INSERTION_MODES
#undef __ENUMERATE_INSERTION_MODE
};
InsertionMode insertion_mode() const { return m_insertion_mode; }
LibWeb: Pass DOM namespace strings as FlyString in more places
2023-11-04 06:19:21 -03:00
static bool is_special_tag(FlyString const& tag_name, Optional<FlyString> const& namespace_);
LibWeb: Improve support for "a" and "li" during "in body" insertion We can now parse welcome.html once again, without resorting to hacks or fallbacks during "in body" :^)
2020-05-29 17:06:05 -03:00
LibWeb: Add basic support for dynamic markup insertion This implements basic support for dynamic markup insertion, adding * Document::open() * Document::write(Vector<String> const&) * Document::writeln(Vector<String> const&) * Document::close() The HTMLParser is modified to make it possible to create a script-created parser which initially only contains a HTMLTokenizer without any data. Aditionally the HTMLParser::run method gains an overload which does not modify the Document and does not run HTMLParser::the_end() so that we can reenter the parser at a later time. Furthermore all FIXMEs that consern the insertion point are implemented wich is defined in the HTMLTokenizer. Additionally the following member-variables of the HTMLParser are now exposed by getter funcions: * m_tokenizer * m_aborted * m_script_nesting_level The HTMLTokenizer is modified so that it contains an insertion point which keeps track of where the next input from the Document::write functions will be inserted. The insertion point is implemented as the charakter offset into m_decoded_input and a boolean describing if the insertion point is defined. Functions to update, check and {re}store the insertion point are also added. The function HTMLTokenizer::insert_eof is added to tell a script-created parser that document::close was called and HTMLParser::the_end() should be called. Lastly an explicit default constructor is added to HTMLTokenizer to create a empty HTMLTokenizer into which data can be inserted.
2022-02-19 11:58:21 -03:00
HTMLTokenizer& tokenizer() { return m_tokenizer; }
LibWeb: Implement aborting the HTML parser This is roughly on-spec, although I had to invent a simple "aborted" state for the tokenizer.
2022-09-20 16:08:14 -03:00
// https://html.spec.whatwg.org/multipage/parsing.html#abort-a-parser
void abort();
LibWeb: Add basic support for dynamic markup insertion This implements basic support for dynamic markup insertion, adding * Document::open() * Document::write(Vector<String> const&) * Document::writeln(Vector<String> const&) * Document::close() The HTMLParser is modified to make it possible to create a script-created parser which initially only contains a HTMLTokenizer without any data. Aditionally the HTMLParser::run method gains an overload which does not modify the Document and does not run HTMLParser::the_end() so that we can reenter the parser at a later time. Furthermore all FIXMEs that consern the insertion point are implemented wich is defined in the HTMLTokenizer. Additionally the following member-variables of the HTMLParser are now exposed by getter funcions: * m_tokenizer * m_aborted * m_script_nesting_level The HTMLTokenizer is modified so that it contains an insertion point which keeps track of where the next input from the Document::write functions will be inserted. The insertion point is implemented as the charakter offset into m_decoded_input and a boolean describing if the insertion point is defined. Functions to update, check and {re}store the insertion point are also added. The function HTMLTokenizer::insert_eof is added to tell a script-created parser that document::close was called and HTMLParser::the_end() should be called. Lastly an explicit default constructor is added to HTMLTokenizer to create a empty HTMLTokenizer into which data can be inserted.
2022-02-19 11:58:21 -03:00
bool aborted() const { return m_aborted; }
LibWeb: Implement aborting the HTML parser This is roughly on-spec, although I had to invent a simple "aborted" state for the tokenizer.
2022-09-20 16:08:14 -03:00
bool stopped() const { return m_stop_parsing; }
LibWeb: Add basic support for dynamic markup insertion This implements basic support for dynamic markup insertion, adding * Document::open() * Document::write(Vector<String> const&) * Document::writeln(Vector<String> const&) * Document::close() The HTMLParser is modified to make it possible to create a script-created parser which initially only contains a HTMLTokenizer without any data. Aditionally the HTMLParser::run method gains an overload which does not modify the Document and does not run HTMLParser::the_end() so that we can reenter the parser at a later time. Furthermore all FIXMEs that consern the insertion point are implemented wich is defined in the HTMLTokenizer. Additionally the following member-variables of the HTMLParser are now exposed by getter funcions: * m_tokenizer * m_aborted * m_script_nesting_level The HTMLTokenizer is modified so that it contains an insertion point which keeps track of where the next input from the Document::write functions will be inserted. The insertion point is implemented as the charakter offset into m_decoded_input and a boolean describing if the insertion point is defined. Functions to update, check and {re}store the insertion point are also added. The function HTMLTokenizer::insert_eof is added to tell a script-created parser that document::close was called and HTMLParser::the_end() should be called. Lastly an explicit default constructor is added to HTMLTokenizer to create a empty HTMLTokenizer into which data can be inserted.
2022-02-19 11:58:21 -03:00
size_t script_nesting_level() const { return m_script_nesting_level; }
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
private:
LibWeb: Change HTMLParser's factory to accept the encoding as StringView No need to force an allocation. This makes a future patch a bit simpler, where we will have the encoding as a String. With this patch, we won't have to convert it to a ByteString.
2024-04-03 22:56:58 -03:00
HTMLParser(DOM::Document&, StringView input, StringView encoding);
LibWeb: Make document.write() work while document is parsing This necessitated making HTMLParser ref-counted, and having it register itself with Document when created. That makes it possible for scripts to add new input at the current parser insertion point. There is now a reference cycle between Document and HTMLParser. This cycle is explicitly broken by calling Document::detach_parser() at the end of HTMLParser::run(). This is a huge progression on ACID3, from 31% to 49%! :^)
2022-02-21 17:54:21 -03:00
HTMLParser(DOM::Document&);
LibWeb: Make the HTMLParser GC-allocated This prevents a reference cycle between a HTMLParser opened via document.open() and the document. It was one of many things keeping some documents alive indefinitely.
2022-10-17 05:46:11 -03:00
virtual void visit_edges(Cell::Visitor&) override;
LibWeb: Store a SpeculativeHTMLParser on the HTML Parser The parser was previously added, but unused. Actually attaching one to the HTML Parser will let us test the limits of Swift interop.
2025-04-04 12:20:27 -03:00
virtual void initialize(JS::Realm&) override;
LibWeb: Make the HTMLParser GC-allocated This prevents a reference cycle between a HTMLParser opened via document.open() and the document. It was one of many things keeping some documents alive indefinitely.
2022-10-17 05:46:11 -03:00
Everywhere: Run clang-format
2022-04-01 14:58:27 -03:00
char const* insertion_mode_name() const;
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
Everywhere: Run clang-format
2022-04-01 14:58:27 -03:00
DOM::QuirksMode which_quirks_mode(HTMLToken const&) const;
LibWeb: Implement quirks mode detection This allows us to determine which mode to render the page in. Exposes "doctype" and "compatMode" on Document. Exposes "name", "publicId" and "systemId" on DocumentType.
2020-07-18 17:17:17 -03:00
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
void handle_initial(HTMLToken&);
void handle_before_html(HTMLToken&);
void handle_before_head(HTMLToken&);
void handle_in_head(HTMLToken&);
void handle_in_head_noscript(HTMLToken&);
void handle_after_head(HTMLToken&);
void handle_in_body(HTMLToken&);
LibWeb: Implement enough HTML parsing to handle a small simple DOM :^) We can now parse a little DOM like this: <!DOCTYPE html> <html> <head></head> <body> <div></div> </body> </html> This is pretty slow work, but the incremental progress is satisfying!
2020-05-23 19:49:22 -03:00
void handle_after_body(HTMLToken&);
void handle_after_after_body(HTMLToken&);
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
void handle_text(HTMLToken&);
LibWeb: Start fleshing out the "in table" parser insertion mode
2020-05-25 15:30:34 -03:00
void handle_in_table(HTMLToken&);
LibWeb: Implement more table support in the new HTML parser This is enough to parse the Google front page! (Note: I did have to hack the tokenizer while parsing Google, in order to avoid named character references screwing everything up. We'll fix that too soon enough!)
2020-05-27 19:27:46 -03:00
void handle_in_table_body(HTMLToken&);
void handle_in_row(HTMLToken&);
void handle_in_cell(HTMLToken&);
LibWeb: Implement more table-related insertion modes
2020-05-30 12:57:41 -03:00
void handle_in_table_text(HTMLToken&);
LibWeb: Support parsing "select" elements (outside of tables)
2020-05-30 14:58:52 -03:00
void handle_in_select_in_table(HTMLToken&);
void handle_in_select(HTMLToken&);
LibWeb: Implement HTML parser "in caption" insertion mode
2020-06-13 01:09:54 -03:00
void handle_in_caption(HTMLToken&);
LibWeb: Implement HTML parser "in column group" insertion mode
2020-06-13 02:22:18 -03:00
void handle_in_column_group(HTMLToken&);
LibWeb: Update parser with more insertion modes :^) Implements handling of InHeadNoScript, InSelectInTable, InTemplate, InFrameset, AfterFrameset, and AfterAfterFrameset.
2020-06-21 01:58:03 -03:00
void handle_in_template(HTMLToken&);
void handle_in_frameset(HTMLToken&);
void handle_after_frameset(HTMLToken&);
void handle_after_after_frameset(HTMLToken&);
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
LibWeb: Add a way to stop the new HTML parser Some things are specced to "stop parsing", which basically just means to stop fetching tokens and jump to "The end"
2020-05-28 13:55:18 -03:00
void stop_parsing() { m_stop_parsing = true; }
LibWeb: Port Element::local_name and TagNames from Deprecated String Which pretty much needs to be done together due to the amount of places where they are compared together. This also involves porting over StackOfOpenElements over to FlyString from DeprecatedFly string to prevent a gazillion calls to `.to_deprecated_fly_string` calls in HTMLParser.
2023-10-01 04:07:44 -03:00
void generate_implied_end_tags(FlyString const& exception = {});
LibWeb: Implement <template> parsing Note that there is currently no way to display them as we can't currently clone nodes. Adds special case for templates for dumping to console. Doesn't add it to the DOM inspector as I'm not sure how to do it.
2020-08-19 18:30:33 -03:00
void generate_all_implied_end_tags_thoroughly();
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
GC::Ref<DOM::Element> create_element_for(HTMLToken const&, Optional<FlyString> const& namespace_, DOM::Node& intended_parent);
LibWeb: Implement more of the foster parenting algorithm in the parser
2020-06-21 12:00:55 -03:00
struct AdjustedInsertionLocation {
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
GC::Ptr<DOM::Node> parent;
GC::Ptr<DOM::Node> insert_before_sibling;
LibWeb: Implement more of the foster parenting algorithm in the parser
2020-06-21 12:00:55 -03:00
};
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
AdjustedInsertionLocation find_appropriate_place_for_inserting_node(GC::Ptr<DOM::Element> override_target = nullptr);
LibWeb: Implement more of the foster parenting algorithm in the parser
2020-06-21 12:00:55 -03:00
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
void insert_an_element_at_the_adjusted_insertion_location(GC::Ref<DOM::Element>);
LibWeb: Parse declarative shadow DOM template elements We now honor the shadowrootmode attribute on template elements while parsing, and instantiate a shadow tree as required by the spec.
2024-06-25 04:43:50 -03:00
LibWeb: Move DOM classes into the Web::DOM namespace LibWeb keeps growing and the Web namespace is filling up fast. Let's put DOM stuff into Web::DOM, just like we already started doing with SVG stuff in Web::SVG.
2020-07-26 14:37:56 -03:00
DOM::Text* find_character_insertion_node();
LibWeb: Buffer text node character insertions in the new parser Instead of appending character-at-a-time, we now buffer character insertions in a StringBuilder, and flush them to the relevant node whenever we start inserting into a new node (and when parsing ends.)
2020-06-03 16:53:08 -03:00
void flush_character_insertions();
LibWeb: Parse declarative shadow DOM template elements We now honor the shadowrootmode attribute on template elements while parsing, and instantiate a shadow tree as required by the spec.
2024-06-25 04:43:50 -03:00
enum class OnlyAddToElementStack {
No,
Yes,
};
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
GC::Ref<DOM::Element> insert_foreign_element(HTMLToken const&, Optional<FlyString> const& namespace_, OnlyAddToElementStack);
GC::Ref<DOM::Element> insert_html_element(HTMLToken const&);
[[nodiscard]] GC::Ptr<DOM::Element> current_node();
[[nodiscard]] GC::Ptr<DOM::Element> adjusted_current_node();
[[nodiscard]] GC::Ptr<DOM::Element> node_before_current_node();
LibWeb: Start implementing character token parsing Now that we've gotten rid of the misguided character buffering in the tokenizer, it actually spits out character tokens that we have to deal with in the parser. This patch implements enough to bring us back to speed with simple.html
2020-05-24 14:51:50 -03:00
void insert_character(u32 data);
LibWeb: Support comments in the "in head" insertion mode
2020-05-24 15:29:01 -03:00
void insert_comment(HTMLToken&);
LibWeb: Start implementing character token parsing Now that we've gotten rid of the misguided character buffering in the tokenizer, it actually spits out character tokens that we have to deal with in the parser. This patch implements enough to bring us back to speed with simple.html
2020-05-24 14:51:50 -03:00
void reconstruct_the_active_formatting_elements();
LibWeb: Yet more work towards parsing www/welcome.html :^)
2020-05-24 17:21:25 -03:00
void close_a_p_element();
LibWeb: Start implementing character token parsing Now that we've gotten rid of the misguided character buffering in the tokenizer, it actually spits out character tokens that we have to deal with in the parser. This patch implements enough to bring us back to speed with simple.html
2020-05-24 14:51:50 -03:00
void process_using_the_rules_for(InsertionMode, HTMLToken&);
LibWeb: Add initial implementation of foreign content parsing Plus sneak in a FIXME for the list of active formatting elements and a test for Element.namespaceURI
2020-10-11 21:51:28 -03:00
void process_using_the_rules_for_foreign_content(HTMLToken&);
LibWeb: Parse enough to handle a <style> inside a <head> :^)
2020-05-24 15:36:43 -03:00
void parse_generic_raw_text_element(HTMLToken&);
LibWeb: A whole bunch of work towards spec-compliant <script> elements This is still very unfinished, but there's at least a skeleton of code.
2020-05-24 17:00:46 -03:00
void increment_script_nesting_level();
void decrement_script_nesting_level();
LibWeb: Assert we're parsing a fragment on fragment cases The specification says that parts labelled as a "fragment case" will only occur when parsing a fragment. It says that if it occurs when not parsing a fragment, then it is a specification error. We should probably assume at this point that it's an implementation error. This fixes a few little mistakes that were caught out by this. Also moves the context element outside insertion mode reset, as other (unimplemented) parts refer to it, such as "adjusted current node". Also cleans up insertion mode reset.
2020-07-21 15:03:05 -03:00
void reset_the_insertion_mode_appropriately();
LibWeb: Handle AAA situation where there's no formatting element found In this case, we're supposed to return from the AAA and then jump to a different behavior in the "in body" insertion mode. So now we do that.
2020-05-30 11:22:25 -03:00
LibWeb: Update parser with more insertion modes :^) Implements handling of InHeadNoScript, InSelectInTable, InTemplate, InFrameset, AfterFrameset, and AfterAfterFrameset.
2020-06-21 01:58:03 -03:00
void adjust_mathml_attributes(HTMLToken&);
LibWeb: Add initial implementation of foreign content parsing Plus sneak in a FIXME for the list of active formatting elements and a test for Element.namespaceURI
2020-10-11 21:51:28 -03:00
void adjust_svg_tag_names(HTMLToken&);
LibWeb: Update parser with more insertion modes :^) Implements handling of InHeadNoScript, InSelectInTable, InTemplate, InFrameset, AfterFrameset, and AfterAfterFrameset.
2020-06-21 01:58:03 -03:00
void adjust_svg_attributes(HTMLToken&);
LibWeb: Port HTMLToken prefix and namespace to Optional<FlyString> Previously these were DeprecatedStrings that contained a null state. After the null state was removed, the nullability of these members was broken. This doesn't seem to cause any problems currently as the HTML parser is not inserting attributes with their full qualified name, but after we fix that problem, this bug surfaces.
2023-11-03 19:26:44 -03:00
static void adjust_foreign_attributes(HTMLToken&);
LibWeb: Update parser with more insertion modes :^) Implements handling of InHeadNoScript, InSelectInTable, InTemplate, InFrameset, AfterFrameset, and AfterAfterFrameset.
2020-06-21 01:58:03 -03:00
LibWeb: Handle AAA situation where there's no formatting element found In this case, we're supposed to return from the AAA and then jump to a different behavior in the "in body" insertion mode. So now we do that.
2020-05-30 11:22:25 -03:00
enum AdoptionAgencyAlgorithmOutcome {
DoNothing,
RunAnyOtherEndTagSteps,
};
AdoptionAgencyAlgorithmOutcome run_the_adoption_agency_algorithm(HTMLToken&);
LibWeb: Implement more table support in the new HTML parser This is enough to parse the Google front page! (Note: I did have to hack the tokenizer while parsing Google, in order to avoid named character references screwing everything up. We'll fix that too soon enough!)
2020-05-27 19:27:46 -03:00
void clear_the_stack_back_to_a_table_context();
void clear_the_stack_back_to_a_table_body_context();
void clear_the_stack_back_to_a_table_row_context();
LibWeb: Parse "td" start tags during "in cell" insertion mode
2020-05-28 06:45:40 -03:00
void close_the_cell();
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
InsertionMode m_insertion_mode { InsertionMode::Initial };
LibWeb: More work on the HTML parser and tokenizer The parser can now switch the state of the tokenizer! Very webby. :^)
2020-05-24 15:24:43 -03:00
InsertionMode m_original_insertion_mode { InsertionMode::Initial };
LibWeb: Start implementing character token parsing Now that we've gotten rid of the misguided character buffering in the tokenizer, it actually spits out character tokens that we have to deal with in the parser. This patch implements enough to bring us back to speed with simple.html
2020-05-24 14:51:50 -03:00
LibWeb: Factor out the "stack of open elements" into its own class This will allow us to write more expressive parsing code. :^)
2020-05-24 14:24:36 -03:00
StackOfOpenElements m_stack_of_open_elements;
LibWeb: Update parser with more insertion modes :^) Implements handling of InHeadNoScript, InSelectInTable, InTemplate, InFrameset, AfterFrameset, and AfterAfterFrameset.
2020-06-21 01:58:03 -03:00
Vector<InsertionMode> m_stack_of_template_insertion_modes;
LibWeb: Implement the first half of the Adoption Agency Algorithm The AAA is a somewhat daunting algorithm you have to run for certain tag when inserted inside the <body> element. The purpose of it is to resolve issues with mismatched tags. This patch implements the first half of the AAA. We also move the "list of active formatting elements" to its own class, since it kept accumulating little behaviors. "Marker" entries are now signified by null Element pointers in the list.
2020-05-27 18:22:42 -03:00
ListOfActiveFormattingElements m_list_of_active_formatting_elements;
LibWeb: Start implementing character token parsing Now that we've gotten rid of the misguided character buffering in the tokenizer, it actually spits out character tokens that we have to deal with in the parser. This patch implements enough to bring us back to speed with simple.html
2020-05-24 14:51:50 -03:00
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
HTMLTokenizer m_tokenizer;
LibWeb: Remember when HTML parser should ignore next line feed character There's a quirk in HTML where the parser should ignore any line feed character immediately following a `pre` or `textarea` start tag. This was working fine when we could peek ahead in the input stream and see the next token, but didn't work in character-at-a-time parsing with document.write(). This commit adds the "can ignore next line feed character" as a parser flag that is maintained across invocations, making it work in this parsing mode as well. 20 new passes in WPT/html/syntax/parsing/ :^)
2025-02-19 12:54:28 -03:00
bool m_next_line_feed_can_be_ignored { false };
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
bool m_foster_parenting { false };
LibWeb: Implement enough HTML parsing to handle a small simple DOM :^) We can now parse a little DOM like this: <!DOCTYPE html> <html> <head></head> <body> <div></div> </body> </html> This is pretty slow work, but the incremental progress is satisfying!
2020-05-23 19:49:22 -03:00
bool m_frameset_ok { true };
bool m_parsing_fragment { false };
LibWeb: Set HTMLParser::m_scripting_enabled as according to the spec This allows <noscript> elements to display their content as proper HTML instead of raw text when scripting is disabled.
2022-09-23 16:43:17 -03:00
// https://html.spec.whatwg.org/multipage/parsing.html#scripting-flag
// The scripting flag is set to "enabled" if scripting was enabled for the Document with which the parser is associated when the parser was created, and "disabled" otherwise.
LibWeb: Parse enough to handle a <style> inside a <head> :^)
2020-05-24 15:36:43 -03:00
bool m_scripting_enabled { true };
LibWeb: Set HTMLParser::m_scripting_enabled as according to the spec This allows <noscript> elements to display their content as proper HTML instead of raw text when scripting is disabled.
2022-09-23 16:43:17 -03:00
LibWeb: A whole bunch of work towards spec-compliant <script> elements This is still very unfinished, but there's at least a skeleton of code.
2020-05-24 17:00:46 -03:00
bool m_invoked_via_document_write { false };
LibWeb: Bring up basic external script execution in the new parser This only works in some narrow cases, but should be enough for our own welcome.html at least. :^)
2020-05-27 18:01:04 -03:00
bool m_aborted { false };
LibWeb: A whole bunch of work towards spec-compliant <script> elements This is still very unfinished, but there's at least a skeleton of code.
2020-05-24 17:00:46 -03:00
bool m_parser_pause_flag { false };
LibWeb: Add a way to stop the new HTML parser Some things are specced to "stop parsing", which basically just means to stop fetching tokens and jump to "The end"
2020-05-28 13:55:18 -03:00
bool m_stop_parsing { false };
LibWeb: A whole bunch of work towards spec-compliant <script> elements This is still very unfinished, but there's at least a skeleton of code.
2020-05-24 17:00:46 -03:00
size_t m_script_nesting_level { 0 };
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
LibWeb+LibJS: Make the EventTarget hierarchy (incl. DOM) GC-allocated This is a monster patch that turns all EventTargets into GC-allocated PlatformObjects. Their C++ wrapper classes are removed, and the LibJS garbage collector is now responsible for their lifetimes. There's a fair amount of hacks and band-aids in this patch, and we'll have a lot of cleanup to do after this.
2022-08-28 08:42:07 -03:00
JS::Realm& realm();
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
GC::Ptr<DOM::Document> m_document;
GC::Ptr<HTMLHeadElement> m_head_element;
GC::Ptr<HTMLFormElement> m_form_element;
GC::Ptr<DOM::Element> m_context_element;
LibWeb: Implement more table-related insertion modes
2020-05-30 12:57:41 -03:00
LibWeb: Store a SpeculativeHTMLParser on the HTML Parser The parser was previously added, but unused. Actually attaching one to the HTML Parser will let us test the limits of Swift interop.
2025-04-04 12:20:27 -03:00
#ifdef LIBWEB_USE_SWIFT
GC::ForeignPtr<Web::SpeculativeHTMLParser> m_speculative_parser;
#endif
LibWeb: Implement more table-related insertion modes
2020-05-30 12:57:41 -03:00
Vector<HTMLToken> m_pending_table_character_tokens;
LibWeb: Buffer text node character insertions in the new parser Instead of appending character-at-a-time, we now buffer character insertions in a StringBuilder, and flush them to the relevant node whenever we start inserting into a new node (and when parsing ends.)
2020-06-03 16:53:08 -03:00
LibGC+Everywhere: Factor out a LibGC from LibJS Resulting in a massive rename across almost everywhere! Alongside the namespace change, we now have the following names: * JS::NonnullGCPtr -> GC::Ref * JS::GCPtr -> GC::Ptr * JS::HeapFunction -> GC::Function * JS::CellImpl -> GC::Cell * JS::Handle -> GC::Root
2024-11-14 12:01:23 -03:00
GC::Ptr<DOM::Text> m_character_insertion_node;
LibWeb: Buffer text node character insertions in the new parser Instead of appending character-at-a-time, we now buffer character insertions in a StringBuilder, and flush them to the relevant node whenever we start inserting into a new node (and when parsing ends.)
2020-06-03 16:53:08 -03:00
StringBuilder m_character_insertion_builder;
LibWeb+LibGC: Import GC swift module into LibWeb and an initial user Start work on a speculative HTML Parser in Swift. This component will walk ahead of the normal HTML parser looking for fetch() requests to make while the normal parser is blocked. This work exposed many holes in the Swift C++ interop component, which have been reported upstream.
2025-03-22 22:12:30 -03:00
} SWIFT_UNSAFE_REFERENCE;
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
LibWeb: Make storage of CSS::StyleValues const-correct Now we consistently use `RefPtr<StyleValue const>` for all StyleValues.
2025-04-15 18:18:27 -03:00
RefPtr<CSS::CSSStyleValue const> parse_dimension_value(StringView);
RefPtr<CSS::CSSStyleValue const> parse_nonzero_dimension_value(StringView);
LibWeb: Remove DOM element deprecated_get_attribute()
2024-01-16 15:04:45 -03:00
Optional<Color> parse_legacy_color_value(StringView);
LibWeb: Move HTML dimension value parsing from CSS to HTML namespace These are part of HTML, not CSS, so let's not confuse things.
2022-03-26 10:29:52 -03:00
LibWeb+LibGC: Import GC swift module into LibWeb and an initial user Start work on a speculative HTML Parser in Swift. This component will walk ahead of the normal HTML parser looking for fetch() requests to make while the normal parser is blocked. This work exposed many holes in the Swift C++ interop component, which have been reported upstream.
2025-03-22 22:12:30 -03:00
// Swift interop
using HTMLParserGCPtr = GC::Ptr<HTMLParser>;
using HTMLParserGCRef = GC::Ref<HTMLParser>;
LibWeb: Start building the tree building part of the new HTML parser This patch adds a new HTMLDocumentParser class. It keeps a tokenizer object internally and feeds itself with one token at a time from it. The names and idioms in this class are expressed as closely to the actual HTML parsing spec as possible, to make development as easy and bug free as possible. :^) This is going to become pretty large, but it's pretty cool!
2020-05-23 19:14:23 -03:00
}
Reference in a new issue Copy permalink