Canvas contexts need a compositor-owned surface path that can be shared
by 2D canvas and WebGL. Add CanvasId and a CanvasSurfaceRegistry, pass
the registry into display-list playback, and teach Skia playback how to
resolve and draw a registered canvas surface.
This only adds the shared display-list command and registry plumbing.
Existing canvas elements still publish their old compositor surfaces, so
the behavior change is left for the later canvas-host commits.
Effect culling can stop a visual context switch after applying
some ancestor contexts. The player left those contexts on the painter
stack even though the target command was skipped.
Restore the painter stack back to the common ancestor when effect
culling aborts the switch.
Display-list replay used to feed compositor-only metadata commands
through the normal painting path. Scroll nodes, wheel hit-test regions,
main-thread wheel regions, and viewport scrollbar metadata do not draw
in the Skia player, but each command still paid visual-context
switching, clip rejection, and no-op dispatch overhead.
Keep those commands in the display list for async scrolling and dumps,
but skip them at the start of painting replay. This keeps the metadata
available to the compositor while removing thousands of no-op paint-side
commands from heavy scrolling pages.
With this change rasterization goes down from 13 ms to 1ms on my
computer for page in
https://github.com/LadybirdBrowser/ladybird/issues/9929
The accumulated visual context tree used index 0 as a null sentinel, so
visual viewport transforms were only represented by adding a normal
transform node when the transform was non-identity. That made callers
treat index 0 as a special no-context value and kept the tree shape
dependent on the visual viewport state.
Reserve index 0 as the visual viewport transform node instead. AVC
traversal, display-list replay, hit testing, debug dumping, and root
paint state now treat that node as a real root. Rebaseline the affected
display-list and async-scrolling text expectations so the explicit root
node appears in AVC dumps.
DisplayListCommandSequence became a thin ByteBuffer wrapper after cached
paint data started storing command bytes directly. It still carried the
command alignment constant and command-header iterator, but it no longer
provided ownership or validation behavior beyond DisplayList itself.
Move the alignment constant and command-header iterator to DisplayList.
Command capture now returns a ByteBuffer directly. PaintableBox installs
cached commands from that byte buffer view while preserving replay and
cache-reference behavior from the consolidated cache holder.
PaintableBox kept an inline array of six optional command-sequence slots
for per-phase paint caching. That made every box pay 336 bytes even when
it never cached paint commands. Cached boxes could allocate a separate
ByteBuffer for each populated phase.
Move storage behind a lazy CachedPaintData holder. The holder records
explicit phase presence and offset/size spans into a single ByteBuffer.
Empty cached phases stay distinct from absent phases. Replay copies
command bytes into the frame display list, and cache references now use
the cached bytes view instead of a retained command object.
On the release arm64 build, sizeof(PaintableBox) drops from 944 to 616
bytes. The old inline holder was 336 bytes; the new inline field is one
8-byte pointer, so uncached boxes are 328 bytes smaller. Cached boxes
now use one consolidated byte buffer for all cached phases.
Cached display list command sequences used to carry their own
DisplayListResourceStorage. That kept resource ID sets and referenced
fonts, images, video frames, and nested display lists alive on every
cached phase, even though the command bytes already contain enough
information to discover those references when they are needed.
This makes cached command sequences store only command bytes. Resource
references are collected transiently from those bytes when a cache entry
is installed or invalidated. The navigable's central display list
resource storage now keeps cache reference counts, so compositor pruning
retains resources used by live cached commands without duplicating
storage in each sequence.
Display lists owned the accumulated visual context tree through a
ref-counted pointer. That tied visual-context state to display-list
lifetime and made compositor updates treat the two as one unit, even
though AVC trees need to become independently updateable compositor
state.
Make accumulated visual context trees plain versioned values, have each
display list store the compatible tree version, and pass the matching
tree alongside display-list updates and replay calls. Replay verifies
that the provided tree matches the display list before executing it.
This prepares the compositor for receiving AVC tree updates separately
from display-list updates: it now accepts the tree as a separate update
parameter, stores it next to the display list, and uses that stored tree
for replay and async-scroll hit testing. Nested display-list resources
carry their own tree snapshots for the same version check.
DisplayListPlayer::execute() used to flush the active painting surface
as part of replay. That made replay and submission inseparable, so
callers could not add extra painting after replay without either
accepting a stale submission boundary or flushing again.
Expose flush(PaintingSurface&) on the player and make execute() only
replay commands. Existing callers now issue an explicit flush at the
same point where the implicit flush used to happen, keeping behavior
unchanged while making the submission boundary visible to compositor
code.
Skip the early effect culling optimization when the draw command has
coordinate-changing visual contexts before the target. Without this, a
filter or opacity layer can be rejected using an untransformed bounding
rect even though a later transform moves the pixels into view.
Add a reference test where an opacity layer starts below the viewport,
then a rotation brings it back on screen.
The helper process needs somewhere to hold the per-context state
CompositorThread currently keeps inside WebContent — display list
and resources, scroll state, async scroll tree, video and compositor
surfaces, backing stores, presentation bookkeeping — before any incoming
IPC has something to dispatch into. CompositorState supplies that, with
two small client interfaces that the next commit plugs the Browser- an
WebContent-side actors into. A few LibWeb painting helpers are exported
with WEB_API so the service can link without dragging in the rest of
the painting code. Nothing references the new class yet.
The compositor IPC path needs to send display-list resources across a
process boundary without losing the resource IDs referenced by
display-list commands. The previous transaction shape carried font and
image-frame objects whose IDs are process-local and would be regenerated
after decoding.
Store explicit IDs with font and image-frame transaction entries and add
IPC serializers that rebuild the resource objects while preserving those
IDs. Fonts now serialize through Gfx::Typeface IPC, image frames use
shareable bitmaps, and video frames keep their existing ID-bearing
transaction entry. This lets the receiver apply the regular transaction
directly.
This is preparatory work required to add IPC between the main and
compositor threads.
Display lists used to own the resource storage needed to replay their
command bytes. That kept the compositor tied to in-process object
ownership: sending a display list update also meant sharing the same
resource container with the recording side.
Move resource storage out of DisplayList and make display list updates
carry a transaction of resources to add and remove. Navigable now tracks
the resources referenced by the current display list, sends only the
delta to the compositor, and trims its recording-side storage to the
active set. The compositor applies those transactions to its own storage
before replacing the cached display list.
This still carries in-process resource objects, but it puts the
ownership boundary in the right place. Command bytes and resource
lifetime are now synchronized explicitly, which is the shape needed
before the compositor can receive serializable resource updates across a
process boundary.
Prep for moving display-list rasterization out of LibWeb by moving the
device-pixel corner radius, corner clip, and anti-aliasing types to
LibGfx.
Keep BorderRadiiData in LibWeb as the CSS-pixel representation and
convert it to Gfx corner radii when recording display-list commands.
There is no need for shared ownership of the resource storage, so make
it a noncopyable movable type owned directly by DisplayList and
DisplayListCommandSequence.
Prep work for serializing display lists across the IPC boundary. Replace
Gfx::Filter's Skia-specific backing with a portable Variant-based
representation, add serialize_filter/deserialize_filter, and store
filter data inline in ApplyEffects/ApplyBackdropFilter command payloads
instead of in DisplayListResourceStorage. FilterResourceId is removed
along with the per-storage filter map.
DisplayListDataSpan offsets used to point into the owning DisplayList's
command byte buffer. That made copied command sequences
position-dependent: whenever commands were captured, replayed, or
appended elsewhere, every inline data span inside command payloads had
to be decoded, adjusted, and written back.
Store those spans relative to the containing command payload instead.
CommandPayloadBuilder still computes padding from the final byte-stream
layout so inline data keeps the alignment expected by typed readers, but
the recorded offset is now local to the payload. This makes command
sequences self-contained byte ranges that can be copied without
rewriting their embedded spans.
Move inline data lookup to DisplayListPlayer by tracking the current
command payload while dispatching a command. The Skia player now
resolves glyph runs, gradient stops, path data, dash arrays, and nested
display list command bytes from that current payload instead of from the
active display list's global byte buffer.
With spans no longer absolute, remove the offset-adjustment helpers from
DisplayList.cpp and simplify append_command_sequence() and
copy_command_sequence_from() to copy command bytes directly. Add small
display-list object byte helpers and tighten the command/header
byte-stream contract to require trivially copyable payload structs.
Paint styles were stored as display list resources, which kept path
painting tied to in-process C++ objects. Move the gradient and pattern
payload into FillPath and StrokePath instead, with gradient stops
serialized as parallel color and position spans.
This is prep work for making display lists serializable across the IPC
boundary without needing to marshal SVG paint style objects separately.
Replace Vector<Variant<...>> display-list storage with a contiguous byte
buffer of command headers, trivially-destructible payloads, and inline
data spans. Variable-size data such as glyphs, paths, dash arrays,
gradient stops, and nested command bytes is appended next to the command
that references it.
The flat representation is prep work for sending display lists over IPC
to a dedicated rasterization process, which the previous Variant-based
structure could not support directly. It also avoids walking command
destructors when a display list is discarded and reduces per-command
allocation and indirection, improving memory use and data locality.
Commands used to keep rendering resources directly in the variant:
image frames, external content and video sources, filters, SVG paint
styles, and nested display lists. That makes the command stream own its
dependencies and prevents it from becoming a POD-like byte buffer.
Add DisplayListResourceStorage and replace those command fields with
stable resource IDs. The storage deduplicates resources by their
existing IDs and can copy only the resources referenced by a captured
command sequence, giving the future IPC boundary a clear list of
resources that must be communicated to the rasterization process.
Add a new visual context node type that negates a scroll frame's offset
during display list replay. This is the inverse of ScrollData: where
ScrollData translates by the scroll offset, ScrollCompensation
translates by the negated offset.
Decoded image data should not continue to traffic in ImmutableBitmap now
that the bitmap wrapper is being retired. Introduce DecodedImageFrame as
the paintable decoded-image unit and store a Bitmap plus ColorSpace in
it directly.
Thread the new frame type through decoded image data, display-list
image commands, filters, canvas drawImage, patterns, WebGL texture
upload, and CSS/SVG image consumers. ImmutableBitmap remains only at
the legacy boundaries that still need it, such as HTML video snapshots
and callers that explicitly ask for a bitmap snapshot.
This keeps color-space ownership with the decoded frame while making
the expensive or legacy ImmutableBitmap path explicit at the few call
sites that still need it.
A video element should record video as video, not as generic external
bitmap content. Add VideoFrameSource and a dedicated display-list
command so the display-list player receives the current
Media::VideoFrame directly.
The Skia player can now upload YUV pixmaps from the frame when a GPU
context is available, without teaching the ordinary ImmutableBitmap
image cache about media formats. If GPU upload is unavailable, the
fallback explicitly converts the frame through YUVData::to_bitmap().
This gives video painting a clear extension point for future frame
backends, such as hardware frames or other planar formats, while
keeping bitmap drawing focused on immutable pixel snapshots.
The rendering thread now uses its own SkiaBackendContext, and the main
thread no longer reaches into GPU-backed Skia objects directly. No Skia
context is shared between threads anymore, so the mutex on
SkiaBackendContext and the lock_context()/unlock_context() pair that
wrapped every PaintingSurface and ImmutableBitmap operation are all dead
weight and can go.
Before applying an effect context (filter, opacity, blend-mode), the
display list player runs a culling check by tentatively switching to the
effect's parent first. When the effect was already on the stack (i.e.
the current context was a descendant of the target) that walk restored
the saveLayer prematurely. The filter was then applied to a partial
batch of commands, and subsequent commands opened a fresh saveLayer for
a second, independent application, producing visibly wrong compositing.
Fold the culling check into switch_to_context: while walking down to
apply contexts, check the bounding rect right before each EffectsData
node and abort if the command is fully clipped. Walks that only go up
never traverse an EffectsData node, so an already-applied effect is
never torn down.
This regressed in cd0705334b.
Cleanup following the per-Navigable rasterization split: since each
Navigable now rasterizes its own display list independently, the HashMap
keyed on display list was always populated with exactly one entry. Pass
the ScrollStateSnapshot directly through the display list player and
rendering thread instead.
Replace per-frame heap-allocated RefCounted ScrollFrame objects with a
single contiguous Vector<ScrollFrame> inside ScrollState. All frames for
a viewport are now stored in one allocation, using type-safe
ScrollFrameIndex instead of RefPtr pointers.
This reduces allocation churn, improves cache locality, and moves
parent-chain traversal (cumulative offset, nearest scrolling ancestor)
into ScrollState — similar to how visual context nodes were recently
consolidated into AccumulatedVisualContextTree.
Replace per-node heap-allocated AtomicRefCounted
AccumulatedVisualContext objects with a single contiguous Vector inside
AccumulatedVisualContextTree. All nodes for a frame are now stored in
one allocation, using type-safe VisualContextIndex instead of RefPtr
pointers.
This reduces allocation churn, improves cache locality, and opens the
door for future snapshotting of visual context state — similar to how
scroll offsets are snapshotted today.
Cache the display list commands produced by each PaintableBox's paint()
on a per-phase basis. On subsequent display list rebuilds, if a
paintable's cache is still valid, replay the recorded commands directly
— skipping paint() and all the property resolution it entails.
Besides saving time on property resolution, this also enables Skia to
reuse path tessellation results across frames — e.g. border paths are
preserved in the cache and don't need to be re-tessellated on every
repaint.
This was arguably put in a worse place by #8162; we mostly need the
device pixel offsets from the scroll state so keep track of those and
convert back to CSS pixels when necessary (i.e. scrollbar data).
Stop converting between CSS and device pixels as part of rendering - the
display list should be as simple as possible, so convert to DevicePixels
once when constructing the display list.
The flush() call at the end of execute_impl() was accidentally left
behind in 2d2af9cd3b. That commit moved flushing into execute(), but
didn't remove the old call from execute_impl(). This caused every
nested display list to trigger a redundant GPU flush.
On an M4 MacBook, this improves Discord from ~65 FPS to 120 FPS.
Add a pre-computed `has_empty_effective_clip` flag on
AccumulatedVisualContext that propagates from parent to child. When a
clip rect or clip path has zero area, all descendant commands are
skipped at display list recording time in `DisplayList::append()`,
so they are never stored or executed.
This allows skipping ~10% of display list commands in the Discord app.
All mask call sites now use saveLayer+DstIn compositing, so the AddMask
command, its SkSL runtime shaders, and CachedRuntimeEffects are no
longer needed.
No callers of draw_painting_surface remain after the previous commits
migrated canvas, video, and SVG to use ExternalContentSource or
ImmutableBitmap snapshots.
Two related problems exist in the current display list architecture:
1. DrawPaintingSurface thread safety: CanvasPaintable::paint() records
the *same* PaintingSurface that the canvas rendering context draws
to. The rendering thread later reads from it, but the main thread
may be concurrently drawing — a data race.
2. Video frames force display list rebuilds: each new video frame
triggers set_needs_display() → full display list rebuild.
Both stem from display list commands holding direct references to
content (surface/bitmap) rather than going through an indirection
layer.
ExternalContentSource is a thread-safe, atomically-refcounted
container that holds an ImmutableBitmap snapshot. The accompanying
DrawExternalContent display list command reads from it during replay,
so producers can swap in new content without rebuilding the list.
Subsequent commits migrate canvas, video, and SVG painting to
ExternalContentSource and then remove DrawPaintingSurface.
By doing so, we attenuate the perspective transform on higher resolution
devices such as Retina displays (2x).
This fixes the perspective transform on sites such as
https://poke-holo.simey.me/.
ApplyTransform is no longer recorded to the display list. Transforms are
now applied inline during display list execution when switching between
accumulated visual contexts.
Change apply_transform to accept parameters directly instead of the
ApplyTransform struct.
Previously, both mask and clip-path were rendered to separate mutable
Gfx::Bitmap objects which forced CPU rasterization. They were then
combined using a CPU pixel-by-pixel operation before being returned
as an ImmutableBitmap.
Instead of including mask in the final bitmap as already rasterized
images, we now use display lists which opens opportunity to utilize
GPU if available.
Bitmap::apply_mask() and ApplyMaskBitmap display list command are no
longer used and have been removed.
When executing display list commands, check if commands with effect
contexts (opacity, filters, blend modes) are outside the viewport
before applying the effect. Since effects don't affect clip state,
would_be_fully_clipped_by_painter() returns the same result before
and after applying effects.
This avoids expensive saveLayer/restore cycles for off-screen commands
with effects like blur, which is particularly beneficial for pages with
many blurred decorative images (e.g., Discord's landing page has 70+
blurred star images).
The optimization only applies when switching to a new effect context,
not for consecutive commands with the same context, to preserve correct
blend mode compositing behavior.
Effects (opacity, blend mode, filters) must be applied in the parent's
coordinate space, before the element's transform. Previously this was
handled by manually switching to the parent's visual context when
applying effects at paint time.
By adding EffectsData to AccumulatedVisualContext and positioning it
before TransformData in the chain, effects are now naturally applied in
the correct order during display list replay, eliminating the special
case in StackingContext::paint().
For SVG filters that can generate content from empty elements (feFlood,
feImage, feTurbulence), a transparent FillRect command is emitted to
trigger the filter through the same AVC pipeline.
Previously, clip-path was applied only during painting in
StackingContext::paint(), which meant hit testing did not respect
clip-path boundaries. Clicks outside the visible clipped region but
inside the element's bounding box would incorrectly register as hits.
By moving clip-path into AccumulatedVisualContext, it becomes part of
the same system that handles transforms, clips, and scroll offsets for
both painting and hit testing, ensuring consistent behavior.
Remove the PushStackingContext and PopStackingContext display list
commands that are no longer used after the AccumulatedVisualContext
integration.
Previously, PushStackingContext was responsible for:
- Applying CSS transforms via its StackingContextTransform field
- Managing opacity layers
- Handling clip paths
- Tracking blend modes
All of this functionality has been replaced by:
- Transform/perspective tracking via AccumulatedVisualContext nodes
- The ApplyEffects command for opacity, blend modes, and filters
- The AddClipPath command for clip paths
