Previously animation ownership was a messy split between
`AnimatedBitmapDecodedImageData` and the consumers (i.e.
`ImageStyleValueResource`, `HTMLImageElement`, and `SVGImageElement`)
with `AnimatedBitmapDecodedImageData` owning the frames and a current
frame index, and the consumers owning the rest of the state (e.g. loop
count, timers to drive the animation forward, their own current index).
This had a couple of main issues:
- While `AnimatedDecodedImageData` partially synchronized animations by
dropping unexpected advancement notifications, this didn't apply to
other animation state which meant, for instance, that a later started
consumer could drive the animation of an earlier one past the max
loop count (albeit without invalidating the earlier consumer).
- Multiple consumers didn't share frame timings, meaning animations
could be up to a full frame out of sync visually.
- Animations were paused depending on whether there were any consumers,
this is different to the behavior in other browsers (where they
continue regardless of whether there are any consumers).
- It was an overgeneralization of how animations need to work - only
`AnimatedBitmapDecodedImageData` works with an indexed frame model,
with animated SVGs (although not yet implemented) relying on their
internal event loop to be driven forward.
Given the above the new approach implemented in this commit is:
- The API for `DecodedImageData` is animation system agnostic, only
exposing `default_frame`, `current_frame`, and `restart_animation`
methods not reliant on providing a specific frame index.
- `AnimatedBitmapDecodedImageData` owns its own timer, loop count,
etc. The animation starts when the first consumer registers and ends
when the document is hidden or becomes inactive (or completes in the
case of finite animations).
- Consumers are invalidated by `AnimatedBitmapDecodedImageData` when
required.
Tests have been added for:
- Animations being paused when the document becomes inactive and
restarted when it becomes active again.
- Frame timings being synchronized across consumers.
- Restarts triggered by `HTMLImageElement` applying to all consumers.
- Processing ending once a non-infinite animation plays to completion.
The tests to ensure animations are cancelled when consumers are removed
(e.g. `animated-background-image-timer-stops-when-hidden.html`) have
been updated to assert the inverse since animation state is now per
resource not per consumer.
This allows us to remove the `frame_rect` accessor.
This also fixes a bug where we computed the scaling mode based on the
clipping rect rather than the draw rect for `ImagePaintable`
WebContent no longer chooses between CPU and GPU display list players,
and the remaining callers always use Skia raster playback. Remove the
PageClient virtual and now-single-value enum, then play SVG image and
cursor display lists directly.
DecodedImageData::paint() used to take both a destination and a
clip rectangle even though most callers passed the same value. SVG
image painting used that API to wrap every nested SVG display list in
save/add-clip/restore, which put an unbounded command in front of
the bounded nested-list command and made offscreen SVG image content
harder to cull.
Move clipping to ImagePaintable, where the object-fit destination can
be compared with the replaced element box. CSS image and marker
painting continue to draw into their destination rect, while repeated
background images keep their explicit tile clip. The scaled decoded
image display-list command now stores only its destination rect and
uses that as its bounds; playback still clips decoded images to that
rect so bitmap rendering stays unchanged.
Cached display list command sequences used to carry their own
DisplayListResourceStorage. That kept resource ID sets and referenced
fonts, images, video frames, and nested display lists alive on every
cached phase, even though the command bytes already contain enough
information to discover those references when they are needed.
This makes cached command sequences store only command bytes. Resource
references are collected transiently from those bytes when a cache entry
is installed or invalidated. The navigable's central display list
resource storage now keeps cache reference counts, so compositor pruning
retains resources used by live cached commands without duplicating
storage in each sequence.
Display lists owned the accumulated visual context tree through a
ref-counted pointer. That tied visual-context state to display-list
lifetime and made compositor updates treat the two as one unit, even
though AVC trees need to become independently updateable compositor
state.
Make accumulated visual context trees plain versioned values, have each
display list store the compatible tree version, and pass the matching
tree alongside display-list updates and replay calls. Replay verifies
that the provided tree matches the display list before executing it.
This prepares the compositor for receiving AVC tree updates separately
from display-list updates: it now accepts the tree as a separate update
parameter, stores it next to the display list, and uses that stored tree
for replay and async-scroll hit testing. Nested display-list resources
carry their own tree snapshots for the same version check.
DisplayListPlayer::execute() used to flush the active painting surface
as part of replay. That made replay and submission inseparable, so
callers could not add extra painting after replay without either
accepting a stale submission boundary or flushing again.
Expose flush(PaintingSurface&) on the player and make execute() only
replay commands. Existing callers now issue an explicit flush at the
same point where the implicit flush used to happen, keeping behavior
unchanged while making the submission boundary visible to compositor
code.
Recording a display list for an SVG image triggered a full layout
update on the SVG's internal document and re-recorded the commands on
every paint. Pages with many small SVG icons therefore could accumulate
tens of thousands of redundant layout updates per page load.
We now cache these display lists per output size to avoid this
redundant work.
The cache currently uses the same 10 entry cap as the cache for
rendered surfaces.
Display lists used to own the resource storage needed to replay their
command bytes. That kept the compositor tied to in-process object
ownership: sending a display list update also meant sharing the same
resource container with the recording side.
Move resource storage out of DisplayList and make display list updates
carry a transaction of resources to add and remove. Navigable now tracks
the resources referenced by the current display list, sends only the
delta to the compositor, and trims its recording-side storage to the
active set. The compositor applies those transactions to its own storage
before replacing the cached display list.
This still carries in-process resource objects, but it puts the
ownership boundary in the right place. Command bytes and resource
lifetime are now synchronized explicitly, which is the shape needed
before the compositor can receive serializable resource updates across a
process boundary.
DecodedImageFrame only wraps a ref-counted Bitmap and color-space
metadata. The frame object itself does not provide shared mutable
state or lifetime ownership beyond those members, so ref-counting it
adds an unnecessary layer of indirection.
Report AudioBuffer channel vectors and SVG decoded image render
caches through the GC external memory hook. Audio sample storage is
already covered by ArrayBuffer accounting.
Decoded image data should not continue to traffic in ImmutableBitmap now
that the bitmap wrapper is being retired. Introduce DecodedImageFrame as
the paintable decoded-image unit and store a Bitmap plus ColorSpace in
it directly.
Thread the new frame type through decoded image data, display-list
image commands, filters, canvas drawImage, patterns, WebGL texture
upload, and CSS/SVG image consumers. ImmutableBitmap remains only at
the legacy boundaries that still need it, such as HTML video snapshots
and callers that explicitly ask for a bitmap snapshot.
This keeps color-space ownership with the decoded frame while making
the expensive or legacy ImmutableBitmap path explicit at the few call
sites that still need it.
ImmutableBitmap still owned the helper that read pixels from a
PaintingSurface and wrapped the result as an ImmutableBitmap. That kept
a surface readback operation attached to the type we are trying to
remove, even though the snapshot is really a property of the painting
surface.
Add PaintingSurface::snapshot_bitmap() as the explicit readback path.
The remaining callers now wrap that bitmap in ImmutableBitmap only at
the places that still need the old abstraction. Canvas serialization
also uses the same helper, so the BGRA8888 premultiplied snapshot
policy has a single owner.
Previously, the active document's lifecycle was bound to
SessionHistoryEntry via DocumentState. The ownership chain was:
Navigable → SessionHistoryEntry → DocumentState → Document
This made it impossible to move SessionHistoryEntry to the UI process
(which cannot own DOM::Document). This commit decouples the two by
giving Navigable a direct m_active_document field that serves as the
authoritative source for active_document().
- Navigable owns m_active_document directly; active_document() reads
from it instead of going through the active session history entry.
- DocumentState no longer holds a Document pointer. Instead, it stores
a document_id for "same document?" checks. Same-document navigations
share a DocumentState and thus the same document_id, while
cross-document navigations create a new DocumentState with a new ID.
- A pending_document parameter is threaded through
finalize_a_cross_document_navigation → apply_the_push_or_replace →
apply_the_history_step so the newly created document reaches
activation without being stored on DocumentState.
- For traversal, the population output delivers the document.
A resolved_document is computed per continuation from either the
pending document, the population output, or the current active
document (for same-document traversals).
When an SVG is loaded as image data (e.g. via <img>), the
XMLDocumentBuilder returns early for decoded SVGs without calling
completely_finish_loading(). This meant that <use> elements which
forward-reference elements parsed after them never got their shadow
trees populated, since:
1. During parsing, the referenced element doesn't exist yet, so
clone_element_tree_as_our_shadow_tree() receives null.
2. The document_completely_loaded callback never fires.
3. update_use_elements_that_reference_this() returns early because
is_completely_loaded() is false.
Fix this by calling completely_finish_loading() on the document after
parsing in SVGDecodedImageData::create().
Use the existing bitmap() method which caches the ImmutableBitmap,
instead of creating a fresh snapshot from the painting surface on
every paint call.
This was regressed by d9e04ec9e8 which replaced draw_painting_surface
with a per-call snapshot to remove the DrawPaintingSurface dependency,
but didn't use the existing bitmap cache.
The last consumer of draw_painting_surface now snapshots the surface
into an ImmutableBitmap and uses draw_scaled_immutable_bitmap instead,
removing the final dependency on DrawPaintingSurface.
SVG images loaded as <img> elements must not execute scripts per spec.
Previously, SVGScriptElement::process_the_script_element() did not
check whether scripting was disabled, so script processing was
triggered via the children_changed() callback during XML parsing,
causing a nested event loop spin.
Fix this by:
- Disabling scripting on the SVG image's Page
- Passing XMLScriptingSupport::Disabled to the XML document builder
- Checking is_scripting_disabled() in SVGScriptElement before
processing any script element
- Logging a diagnostic when SVG XML parsing fails (previously the
parse result was silently discarded)
For XHTML documents, resolve named character entities (e.g., )
using the HTML entity table via a getEntity SAX callback. This avoids
parsing a large embedded DTD on every document and matches the approach
used by Blink and WebKit.
This also removes the now-unused DTD infrastructure:
- Remove resolve_external_resource callback from Parser::Options
- Remove resolve_xml_resource() function and its ~60KB embedded DTD
- Remove all call sites passing the unused callback
This allows us to render SVGs on the GPU even when requesting a bitmap.
Since SVG bitmaps are an ImmutableBitmap, it can store a painting
surface snapshot, so we don't need to render bitmaps on the CPU.
Fixes Shopify homepage freezing for ~20 seconds rendering an SVG
background-image on the CPU on my machine.
Instead of painting DecodedImageData by first asking it for a bitmap
and then painting that, this commit adds two new APIs:
- frame_rect(frame_index):
Gets the size of the animation frame at the given index.
- paint(context, ...):
Paints the DecodedImageData into a DisplayListRecordingContext.
The main powerful thing here is that this allows SVGDecodedImageData
to render itself using the GPU when available.
We were already using the XML parser for SVG files when opening at the
top level. This patch makes things consistent by using the same code
path when parsing SVG-as-image.
We also make some tweaks in SVG presentation attribute handling since
we can no longer rely on the HTML length attribute parsing quirk when
parsing width/height attributes.
PaintContext dates back to a time when display lists didn't exist and it
truly represented "paint context". Renaming it to better align with its
current role.
6507d23 introduced a bug when snapshot for iframe is saved in
`PaintNestedDisplayList` and, since display lists are immutable, it's
not possible to update before the next repaint.
This change fixes the issue by moving `ScrollStateSnapshot` for
nested display lists from `PaintNestedDisplayList` to
`HashMap<NonnullRefPtr<DisplayList>, ScrollStateSnapshot>` that is
placed into pending rendering task, making it possible to update
snapshots for all display lists before the next repaint.
This change doesn't have a test because it's really hard to make a ref
test that will specifically check scenario when scroll offset of an
iframe is advanced after display list is cached. We already have
`Tests/LibWeb/Ref/input/scroll-iframe.html` but unfortunately it did
not catch this bug.
Fixes https://github.com/LadybirdBrowser/ladybird/issues/5486
Previously, an SVG with width of zero would have am intrinsic aspect
ratio of zero. With this change, if an SVG has a width or height of
zero, the intrinsic aspect ratio is determined by the SVG's viewbox.
Instead, porting over all users to use the newly created
Origin::create_opaque factory function. This also requires porting
over some users of Origin to avoid default construction.
These are not associated with a javascript realm, so to avoid
confusion about which realm these need to be created in, make
all of these objects a GC::Cell, and deal with the fallout.
With this change we save a copy of of scroll state at the time of
recording a display list, instead of actual ScrollState pointer that
could be modifed by the main thread while display list is beings
rasterized on the rendering thread, which leads to a frame painted with
inconsistent scroll state.
Fixes https://github.com/LadybirdBrowser/ladybird/issues/4288
Deleteing set_surface() makes DisplayListPlayer API a bit more intuitive
because now caller doesn't have to think whether it's necessary to
restore previous surface after execution, instead DisplayListPlayer
takes care of it by maintaining a stack of surfaces.
This reduces the number of `.cpp` files that need to be recompiled when
one of the below header files changes as follows:
CSS/ComputedProperties.h: 1113 -> 49
CSS/ComputedValues.h: 1120 -> 209
This is required to store Content Security Policies, as their
Directives are implemented as subclasses with overridden virtual
functions. Thus, they cannot be stored as generic Directive classes, as
it'll lose the ability to call overridden functions when they are
copied.
We were previously crashing when invoking 'scroll to the fragment' on
such documents as it was unable to find the active document. This is
as a result of our AD-HOC implementation not setting up the document
fully to mark it is active before running the parser.
Fixes a crash on https://tweakers.net.
Resulting in a massive rename across almost everywhere! Alongside the
namespace change, we now have the following names:
* JS::NonnullGCPtr -> GC::Ref
* JS::GCPtr -> GC::Ptr
* JS::HeapFunction -> GC::Function
* JS::CellImpl -> GC::Cell
* JS::Handle -> GC::Root
Now that the heap has no knowledge about a JavaScript realm and is
purely for managing the memory of the heap, it does not make sense
to name this function to say that it is a non-realm variant.
The main motivation behind this is to remove JS specifics of the Realm
from the implementation of the Heap.
As a side effect of this change, this is a bit nicer to read than the
previous approach, and in my opinion, also makes it a little more clear
that this method is specific to a JavaScript Realm.
