Previously animation ownership was a messy split between
`AnimatedBitmapDecodedImageData` and the consumers (i.e.
`ImageStyleValueResource`, `HTMLImageElement`, and `SVGImageElement`)
with `AnimatedBitmapDecodedImageData` owning the frames and a current
frame index, and the consumers owning the rest of the state (e.g. loop
count, timers to drive the animation forward, their own current index).
This had a couple of main issues:
- While `AnimatedDecodedImageData` partially synchronized animations by
dropping unexpected advancement notifications, this didn't apply to
other animation state which meant, for instance, that a later started
consumer could drive the animation of an earlier one past the max
loop count (albeit without invalidating the earlier consumer).
- Multiple consumers didn't share frame timings, meaning animations
could be up to a full frame out of sync visually.
- Animations were paused depending on whether there were any consumers,
this is different to the behavior in other browsers (where they
continue regardless of whether there are any consumers).
- It was an overgeneralization of how animations need to work - only
`AnimatedBitmapDecodedImageData` works with an indexed frame model,
with animated SVGs (although not yet implemented) relying on their
internal event loop to be driven forward.
Given the above the new approach implemented in this commit is:
- The API for `DecodedImageData` is animation system agnostic, only
exposing `default_frame`, `current_frame`, and `restart_animation`
methods not reliant on providing a specific frame index.
- `AnimatedBitmapDecodedImageData` owns its own timer, loop count,
etc. The animation starts when the first consumer registers and ends
when the document is hidden or becomes inactive (or completes in the
case of finite animations).
- Consumers are invalidated by `AnimatedBitmapDecodedImageData` when
required.
Tests have been added for:
- Animations being paused when the document becomes inactive and
restarted when it becomes active again.
- Frame timings being synchronized across consumers.
- Restarts triggered by `HTMLImageElement` applying to all consumers.
- Processing ending once a non-infinite animation plays to completion.
The tests to ensure animations are cancelled when consumers are removed
(e.g. `animated-background-image-timer-stops-when-hidden.html`) have
been updated to assert the inverse since animation state is now per
resource not per consumer.
This allows us to remove the `frame_rect` accessor.
This also fixes a bug where we computed the scaling mode based on the
clipping rect rather than the draw rect for `ImagePaintable`
Previously we used the `frame_rect` size, this is the same as the
intrinsic size for bitmap images but is `OptionalNone` for SVG which
caused us to always fall back to the `image_rect` and thus not apply
any scaling for SVGs regardless of whether they had intrinsic sizing.
DecodedImageData::paint() used to take both a destination and a
clip rectangle even though most callers passed the same value. SVG
image painting used that API to wrap every nested SVG display list in
save/add-clip/restore, which put an unbounded command in front of
the bounded nested-list command and made offscreen SVG image content
harder to cull.
Move clipping to ImagePaintable, where the object-fit destination can
be compared with the replaced element box. CSS image and marker
painting continue to draw into their destination rect, while repeated
background images keep their explicit tile clip. The scaled decoded
image display-list command now stores only its destination rect and
uses that as its bounds; playback still clips decoded images to that
rect so bitmap rendering stays unchanged.
When image data becomes available, intrinsic dimensions can change, but
that does not require relayout for image elements whose computed width,
height, and min/max constraints are independent of intrinsic sizing. In
that case, invalidate paint and intrinsic-size caches instead of marking
layout dirty.
Keep the conservative relayout path for auto, percentage,
calc-percentage, and intrinsic sizing constraints, since decoded image
dimensions can affect used size in indefinite contexts. Query image
availability while painting so an existing paintable can switch from alt
text to image content without relayout.
Add text coverage that waits for image load events and checks auto,
percentage, calc-percentage, intrinsic keyword, min/max percentage, and
fixed-size cases without relying on timeouts.
The Paintable tree and its supplemental painting data structures were
GC allocated because that was the easiest way to manage it and avoid
leaks introduced by ref cycles. This included the Paintable subclasses
themselves plus StackingContext, ChromeWidget, Scrollbar, ResizeHandle,
and scroll-frame state.
We are now trying to reduce GC allocation churn on layout and painting
updates, so keeping this short-lived rendering tree outside the JS heap
is a better fit. Move Paintable to RefCountedTreeNode, make painting
helpers ref-counted or weakly reference Paintables, and update the
layout and event-handler call sites to use RefPtr/WeakPtr ownership.
DecodedImageFrame now owns decoded bitmap pixels directly, so the
separate ImmutableBitmap wrapper no longer carries useful semantics.
Remove the class and pass decoded image frames or bitmaps at the
boundaries where pixels are actually required.
The Skia image cache now keys off DecodedImageFrame, matching the
display-list commands that paint decoded images. Video frames stay
owned by LibMedia, with the explicit YUV-to-bitmap conversion living
at HTMLVideoElement's decoded-frame entry point for canvas and WebGL
callers.
Other browsers also respect this and there's nothing in the spec to
suggest this shouldn't happen. I removed the border since it was
inconsistently drawn only on missing images with alt text.
Other browsers do not have this at all.
Neither of these classes did anything useful as ViewportClients:
- ImagePaintable called set_visible_in_viewport() which is a FIXME
no-op in every ImageProvider implementation.
- VideoBox had an empty did_set_viewport_rect() with a FIXME comment.
More importantly, they caused a crash when a DOM node was adopted into
a different document: the old ImagePaintable/VideoBox would still be
registered with the old document, but their document() accessor (which
goes through the DOM node) would return the new document. During GC
finalization, unregister_viewport_client() would fail because it was
trying to unregister from the wrong document.
The only meaningful ViewportClient is HTMLImageElement (for responsive
image source selection), which already handles document adoption
correctly in adopted_from().
Fixes crash when loading https://msn.com/
Remove 11 heavy includes from Document.h that were only needed for
pointer/reference types (already forward-declared in Forward.h), and
extract the nested ViewportClient interface to a standalone header.
This reduces Document.h's recompilation cascade from ~1228 files to
~717 files (42% reduction). Headers like BrowsingContext.h that were
previously transitively included see even larger improvements (from
~1228 down to ~73 dependents).
Previously, text selection always used the system highlight color. This
implements support for the ::selection pseudo-element's background-color
and color properties.
For form controls like <input> and <textarea>, the selection style is
looked up on the shadow host element, since the actual text lives inside
their shadow DOM.
The text painting logic has been refactored to split fragments into
styled spans (before selection, selected, after selection) so that each
portion can be rendered with its appropriate colors, taking care not to
allocate in 99%+ of fragment rendering cases.
Reuse existing paintables during relayout to reduce GC allocation
pressure. Each paintable subclass implements reset_for_relayout()
to clear state before reuse.
Currently, ImageProvider::current_image_bitmap takes a Gfx::IntSize
argument which determines the size of the returned bitmap. The default
value of this argument is 0x0 which causes the function to return
nullptr. This behavior is evidently unintuitive enough that it has lead
to incorrect usage in multiple places. For example, the 2D canvas
drawImage method will never actually draw anything because it calls
current_image_bitmap with no arguments. And the naturalWidth and
naturalHeight of an image will always return 0 (even after the image has
loaded) for the same reason.
To correct this and hopefully avoid similar issues in the future,
ImageProvider::current_image_bitmap will be renamed to
current_image_bitmap_sized, and the default value for the size argument
will be removed. For consistency, a similar change will be made to
SVGImageElement::default_image_bitmap.
The existing current_image_bitmap function will no longer take a size
argument. Instead it will always return a bitmap of the image's
intrinsic size. This seems to be what most existing callers had already
assumed was the function's behavior.
PaintContext dates back to a time when display lists didn't exist and it
truly represented "paint context". Renaming it to better align with its
current role.
Own clip rect is alredy applied in `PaintableBox::before_paint()` for
all image paintables, so there's no need to do it once again in
`ImagePaintable::paint()`.
Instead of trying to manually determine which parts of a bitmap fall
within the box of the `<img>` element, just draw the whole bitmap and
let Skia clip the draw-area to the correct rectangle.
This fixes a bug where the entire bitmap was squashed into the rectangle
of the image box instead of being clipped.
With this change, image rendering is now correct enough to import some
of the WPT tests for object-fit and object-position. To get some good
coverage I have imported all tests for the `<img>` tag. I also wanted to
import a subset of the tests for the `<object>` tag, since those are
passing as well now. Unfortunately, they are flaky for unknown reasons.
This is the second attempt at this bugfix. The prior one was e055927ead
and broke image rendering whenever the page was scrolled. It has
subsequently been reverted in 16b14273d1. Hopefully this time it is not
horribly broken.
Instead of trying to manually determine which parts of a bitmap fall
within the box of the `<img>` element, just draw the whole bitmap and
let Skia clip the draw-area to the correct rectangle.
This fixes a bug where the entire bitmap was squashed into the rectangle
of the image box instead of being clipped.
With this change, image rendering is now correct enough to import some
of the WPT tests for object-fit and object-position. To get some good
coverage I have imported all tests for the `<img>` tag. I also wanted to
import a subset of the tests for the `<object>` tag, since those are
passing as well now. Unfortunately, they are flaky for unknown reasons.
This reduces the number of `.cpp` files that need to be recompiled when
one of the below header files changes as follows:
Painting/Command.h: 1030 -> 61
Painting/DisplayList.h: 1030 -> 60
Painting/DisplayListRecorder.h: 557 -> 59
Previously most of the calculations for `object-fit` and
`object-position` were based on device pixels, meaning that images would
render differently based on zoom and DPI settings. Instead those
calculations now use css pixels and only the final draw-call is based
on device-pixels.
Resulting in a massive rename across almost everywhere! Alongside the
namespace change, we now have the following names:
* JS::NonnullGCPtr -> GC::Ref
* JS::GCPtr -> GC::Ptr
* JS::HeapFunction -> GC::Function
* JS::CellImpl -> GC::Cell
* JS::Handle -> GC::Root
Now that the heap has no knowledge about a JavaScript realm and is
purely for managing the memory of the heap, it does not make sense
to name this function to say that it is a non-realm variant.
