Nested navigables were represented through compositor surface ids owned
by the parent context. That forced CompositorState and ContextState to
maintain bidirectional attach/detach bookkeeping, publish child
snapshots into a surface map, and keep presentation mode variants just
to distinguish UI presentation from parent composition.
Record the child compositor context id directly in the display list and
let the compositor resolve it against the painting parent at playback
time. Child contexts now keep their parent context id and latest
rendered surface, while parents no longer track child maps or compositor
surface ids. UI presentation is represented separately from parent
composition, so closing a page only stops client presentation and nested
contexts keep using set_parent_context.
Canvas contexts need a compositor-owned surface path that can be shared
by 2D canvas and WebGL. Add CanvasId and a CanvasSurfaceRegistry, pass
the registry into display-list playback, and teach Skia playback how to
resolve and draw a registered canvas surface.
This only adds the shared display-list command and registry plumbing.
Existing canvas elements still publish their old compositor surfaces, so
the behavior change is left for the later canvas-host commits.
DecodedImageData::paint() used to take both a destination and a
clip rectangle even though most callers passed the same value. SVG
image painting used that API to wrap every nested SVG display list in
save/add-clip/restore, which put an unbounded command in front of
the bounded nested-list command and made offscreen SVG image content
harder to cull.
Move clipping to ImagePaintable, where the object-fit destination can
be compared with the replaced element box. CSS image and marker
painting continue to draw into their destination rect, while repeated
background images keep their explicit tile clip. The scaled decoded
image display-list command now stores only its destination rect and
uses that as its bounds; playback still clips decoded images to that
rect so bitmap rendering stays unchanged.
PaintNestedDisplayList carried both a display list resource id and an
inline copy of that resource's command buffer. Repeated SVG image paints
therefore duplicated the nested command stream in every parent display
list even though resource storage already owns and deduplicates it.
Drop the inline command byte span and replay nested display lists from
the resource table. Resource collection now follows the referenced
resource's command bytes as well, so cached paint data and compositor
transactions keep the same nested resource retention behavior.
Fixes https://github.com/LadybirdBrowser/ladybird/issues/9929
DisplayListRecorder::replay_cached_commands() always entered the
display-list decoder to update save nesting, even when a paintable had
no cached commands. Empty command caches are common on the
ViewportPaintable benchmark, so return before decoding when the cache
is empty.
DrawGlyphRun display-list commands serialized Gfx::DrawGlyph values
though replay only needs each glyph's position and glyph id. This kept
text layout-only fields in the command stream for every glyph and made
glyph-heavy pages spend most of their display-list bytes on unused data.
Serialize a display-list-specific glyph payload containing only the
position and glyph id, and have the Skia player read that compact form.
The same helper is used for text shadows, which replay through
DrawGlyphRun as well. On the Hacker News item trace this reduces the
stored glyph array from 24 to 12 bytes per glyph.
Every compositor wheel hit-test target carried a Gfx::CornerRadii
payload even though most targets have square corners. On YouTube this
made wheel hit-test targets the largest display-list bucket, with
thousands of targets paying for empty radii.
Keep CompositorWheelHitTestTarget as the compact square-corner command
and add CompositorWheelHitTestTargetWithCornerRadii for the rounded
case. Painting emits the larger command only when the computed corner
radii are non-empty, and async scrolling normalizes both payloads back
into the same target state.
A YouTube recording with 5,483 square-corner wheel hit-test targets used
80-byte compact commands instead of the 112-byte rounded command,
reducing the top-level display list by about 175 KiB for that page.
DisplayListCommandSequence became a thin ByteBuffer wrapper after cached
paint data started storing command bytes directly. It still carried the
command alignment constant and command-header iterator, but it no longer
provided ownership or validation behavior beyond DisplayList itself.
Move the alignment constant and command-header iterator to DisplayList.
Command capture now returns a ByteBuffer directly. PaintableBox installs
cached commands from that byte buffer view while preserving replay and
cache-reference behavior from the consolidated cache holder.
PaintableBox kept an inline array of six optional command-sequence slots
for per-phase paint caching. That made every box pay 336 bytes even when
it never cached paint commands. Cached boxes could allocate a separate
ByteBuffer for each populated phase.
Move storage behind a lazy CachedPaintData holder. The holder records
explicit phase presence and offset/size spans into a single ByteBuffer.
Empty cached phases stay distinct from absent phases. Replay copies
command bytes into the frame display list, and cache references now use
the cached bytes view instead of a retained command object.
On the release arm64 build, sizeof(PaintableBox) drops from 944 to 616
bytes. The old inline holder was 336 bytes; the new inline field is one
8-byte pointer, so uncached boxes are 328 bytes smaller. Cached boxes
now use one consolidated byte buffer for all cached phases.
Cached display list command sequences used to carry their own
DisplayListResourceStorage. That kept resource ID sets and referenced
fonts, images, video frames, and nested display lists alive on every
cached phase, even though the command bytes already contain enough
information to discover those references when they are needed.
This makes cached command sequences store only command bytes. Resource
references are collected transiently from those bytes when a cache entry
is installed or invalidated. The navigable's central display list
resource storage now keeps cache reference counts, so compositor pruning
retains resources used by live cached commands without duplicating
storage in each sequence.
Display lists owned the accumulated visual context tree through a
ref-counted pointer. That tied visual-context state to display-list
lifetime and made compositor updates treat the two as one unit, even
though AVC trees need to become independently updateable compositor
state.
Make accumulated visual context trees plain versioned values, have each
display list store the compatible tree version, and pass the matching
tree alongside display-list updates and replay calls. Replay verifies
that the provided tree matches the display list before executing it.
This prepares the compositor for receiving AVC tree updates separately
from display-list updates: it now accepts the tree as a separate update
parameter, stores it next to the display list, and uses that stored tree
for replay and async-scroll hit testing. Nested display-list resources
carry their own tree snapshots for the same version check.
Future compositor-process work needs to push video frame updates without
going through VideoFrameSource. That object cannot be shared through
IPC, so video display-list resources now use stable VideoFrameResourceId
values and the current frame is sent through explicit resource/update
commands.
Display lists kept canvas and nested navigable content alive through
ExternalContentSource objects. That made the resource graph depend on
process-local object identity instead of a stable surface handle, which
blocks compositor process isolation and made teardown-sensitive embedded
content harder to reason about.
Allocate CompositorSurfaceId values for canvases and child navigables,
publish their backing stores to the owning compositor, and paint them
with DrawCompositorSurface. Child navigables now publish to parent
compositors by CompositorContextId instead of raw object pointers, so
the in-process path uses the same stable addressing model required by a
remote compositor.
Clear and skip stale child surfaces during teardown, preserve Skia
canvas state while drawing compositor surfaces, and add display-list
coverage for canvas and iframe compositor surfaces. The nested navigable
async-scrolling baseline now expects DrawCompositorSurface.
Display lists used to own the resource storage needed to replay their
command bytes. That kept the compositor tied to in-process object
ownership: sending a display list update also meant sharing the same
resource container with the recording side.
Move resource storage out of DisplayList and make display list updates
carry a transaction of resources to add and remove. Navigable now tracks
the resources referenced by the current display list, sends only the
delta to the compositor, and trims its recording-side storage to the
active set. The compositor applies those transactions to its own storage
before replacing the cached display list.
This still carries in-process resource objects, but it puts the
ownership boundary in the right place. Command bytes and resource
lifetime are now synchronized explicitly, which is the shape needed
before the compositor can receive serializable resource updates across a
process boundary.
Async wheel routing used scroll node scrollports as the hit-test
surface. That works while the scrollable content is the topmost content
under the pointer, but it ignores the rest of the painted scene. A
sibling with a higher z-index can visually cover a nested scroller
without being part of that scroller's subtree. In that case the
compositor could still pick the covered scroller from its scrollport
rect and consume the wheel event, even though normal hit testing would
target the covering element and let the page or an ancestor handle the
scroll.
Record wheel hit-test targets as display-list metadata in paint order
and resolve each target to the scroll frame that should receive wheel
deltas. The compositor can then walk those targets from front to back,
preserve non-passive wheel regions as main-thread barriers, and only
scroll the node associated with the topmost hit-testable box.
Use compositor hit-test commands in the display list to rebuild async
wheel targets, and serialize the async scroll metadata needed to
reconstruct AsyncScrollingState from the same display-list snapshot.
Driving the async scroll tree off the display list rather than a
separately collected tree has a few benefits:
- No additional full paintable tree traversal is required, since the
information needed by the compositor is gathered while recording
the display list.
- The display list is already serializable, so the async scroll tree
no longer needs its own serialization path.
- It is more debuggable, as the existing display list dump now also
covers the data used to reconstruct the async scroll tree.
- In the future we will want to include other areas that can
interfere with hit-testing; recording them during display list
construction makes it straightforward to preserve a hit-testing
order that matches painting order.
Prep for moving display-list rasterization out of LibWeb by moving the
device-pixel corner radius, corner clip, and anti-aliasing types to
LibGfx.
Keep BorderRadiiData in LibWeb as the CSS-pixel representation and
convert it to Gfx corner radii when recording display-list commands.
Prep for moving display-list rasterization out of LibWeb. Display-list
gradient commands should not expose CSS enum types to the playback side,
so add the matching interpolation method types to LibGfx and convert CSS
values while recording.
Prep work for serializing display lists across the IPC boundary. Replace
Gfx::Filter's Skia-specific backing with a portable Variant-based
representation, add serialize_filter/deserialize_filter, and store
filter data inline in ApplyEffects/ApplyBackdropFilter command payloads
instead of in DisplayListResourceStorage. FilterResourceId is removed
along with the per-storage filter map.
DisplayListDataSpan offsets used to point into the owning DisplayList's
command byte buffer. That made copied command sequences
position-dependent: whenever commands were captured, replayed, or
appended elsewhere, every inline data span inside command payloads had
to be decoded, adjusted, and written back.
Store those spans relative to the containing command payload instead.
CommandPayloadBuilder still computes padding from the final byte-stream
layout so inline data keeps the alignment expected by typed readers, but
the recorded offset is now local to the payload. This makes command
sequences self-contained byte ranges that can be copied without
rewriting their embedded spans.
Move inline data lookup to DisplayListPlayer by tracking the current
command payload while dispatching a command. The Skia player now
resolves glyph runs, gradient stops, path data, dash arrays, and nested
display list command bytes from that current payload instead of from the
active display list's global byte buffer.
With spans no longer absolute, remove the offset-adjustment helpers from
DisplayList.cpp and simplify append_command_sequence() and
copy_command_sequence_from() to copy command bytes directly. Add small
display-list object byte helpers and tighten the command/header
byte-stream contract to require trivially copyable payload structs.
Paint styles were stored as display list resources, which kept path
painting tied to in-process C++ objects. Move the gradient and pattern
payload into FillPath and StrokePath instead, with gradient stops
serialized as parallel color and position spans.
This is prep work for making display lists serializable across the IPC
boundary without needing to marshal SVG paint style objects separately.
Replace Vector<Variant<...>> display-list storage with a contiguous byte
buffer of command headers, trivially-destructible payloads, and inline
data spans. Variable-size data such as glyphs, paths, dash arrays,
gradient stops, and nested command bytes is appended next to the command
that references it.
The flat representation is prep work for sending display lists over IPC
to a dedicated rasterization process, which the previous Variant-based
structure could not support directly. It also avoids walking command
destructors when a display list is discarded and reduces per-command
allocation and indirection, improving memory use and data locality.
Commands used to keep rendering resources directly in the variant:
image frames, external content and video sources, filters, SVG paint
styles, and nested display lists. That makes the command stream own its
dependencies and prevents it from becoming a POD-like byte buffer.
Add DisplayListResourceStorage and replace those command fields with
stable resource IDs. The storage deduplicates resources by their
existing IDs and can copy only the resources referenced by a captured
command sequence, giving the future IPC boundary a clear list of
resources that must be communicated to the rasterization process.
DecodedImageFrame only wraps a ref-counted Bitmap and color-space
metadata. The frame object itself does not provide shared mutable
state or lifetime ownership beyond those members, so ref-counting it
adds an unnecessary layer of indirection.
Decoded image data should not continue to traffic in ImmutableBitmap now
that the bitmap wrapper is being retired. Introduce DecodedImageFrame as
the paintable decoded-image unit and store a Bitmap plus ColorSpace in
it directly.
Thread the new frame type through decoded image data, display-list
image commands, filters, canvas drawImage, patterns, WebGL texture
upload, and CSS/SVG image consumers. ImmutableBitmap remains only at
the legacy boundaries that still need it, such as HTML video snapshots
and callers that explicitly ask for a bitmap snapshot.
This keeps color-space ownership with the decoded frame while making
the expensive or legacy ImmutableBitmap path explicit at the few call
sites that still need it.
A video element should record video as video, not as generic external
bitmap content. Add VideoFrameSource and a dedicated display-list
command so the display-list player receives the current
Media::VideoFrame directly.
The Skia player can now upload YUV pixmaps from the frame when a GPU
context is available, without teaching the ordinary ImmutableBitmap
image cache about media formats. If GPU upload is unavailable, the
fallback explicitly converts the frame through YUVData::to_bitmap().
This gives video painting a clear extension point for future frame
backends, such as hardware frames or other planar formats, while
keeping bitmap drawing focused on immutable pixel snapshots.
Replace per-frame heap-allocated RefCounted ScrollFrame objects with a
single contiguous Vector<ScrollFrame> inside ScrollState. All frames for
a viewport are now stored in one allocation, using type-safe
ScrollFrameIndex instead of RefPtr pointers.
This reduces allocation churn, improves cache locality, and moves
parent-chain traversal (cumulative offset, nearest scrolling ancestor)
into ScrollState — similar to how visual context nodes were recently
consolidated into AccumulatedVisualContextTree.
Replace per-node heap-allocated AtomicRefCounted
AccumulatedVisualContext objects with a single contiguous Vector inside
AccumulatedVisualContextTree. All nodes for a frame are now stored in
one allocation, using type-safe VisualContextIndex instead of RefPtr
pointers.
This reduces allocation churn, improves cache locality, and opens the
door for future snapshotting of visual context state — similar to how
scroll offsets are snapshotted today.
Move shadow rect computation, offset translation, and corner radii
spread-distance adjustment from the Skia display list player to the
recording site in ShadowPainting.cpp. This avoids recomputing derived
geometry every time the display list is replayed.
Replace PaintBoxShadowParams with precomputed fields directly on
PaintOuterBoxShadow and PaintInnerBoxShadow command structs. Add
CornerRadii::adjust_corners_for_spread_distance() to centralize the
CSS spec's spread-distance-to-radius adjustment logic.
Cache the display list commands produced by each PaintableBox's paint()
on a per-phase basis. On subsequent display list rebuilds, if a
paintable's cache is still valid, replay the recorded commands directly
— skipping paint() and all the property resolution it entails.
Besides saving time on property resolution, this also enables Skia to
reuse path tessellation results across frames — e.g. border paths are
preserved in the cache and don't need to be re-tessellated on every
repaint.
Stop converting between CSS and device pixels as part of rendering - the
display list should be as simple as possible, so convert to DevicePixels
once when constructing the display list.
This applies the same pattern used for background-clip: text (commit
f2e6f70fbb).
Results in visible performance improvement in Discord app where
previously, according to profiles, we spent lots of time allocating
surfaces for masks.
No callers of draw_painting_surface remain after the previous commits
migrated canvas, video, and SVG to use ExternalContentSource or
ImmutableBitmap snapshots.
Two related problems exist in the current display list architecture:
1. DrawPaintingSurface thread safety: CanvasPaintable::paint() records
the *same* PaintingSurface that the canvas rendering context draws
to. The rendering thread later reads from it, but the main thread
may be concurrently drawing — a data race.
2. Video frames force display list rebuilds: each new video frame
triggers set_needs_display() → full display list rebuild.
Both stem from display list commands holding direct references to
content (surface/bitmap) rather than going through an indirection
layer.
ExternalContentSource is a thread-safe, atomically-refcounted
container that holds an ImmutableBitmap snapshot. The accompanying
DrawExternalContent display list command reads from it during replay,
so producers can swap in new content without rebuilding the list.
Subsequent commits migrate canvas, video, and SVG painting to
ExternalContentSource and then remove DrawPaintingSurface.
Previously we would resolve font features
(https://drafts.csswg.org/css-fonts-4/#feature-variation-precedence)
per element, while this works for the current subset of the font feature
resolution algorithm that we support, some as yet unimplemented parts
require us to know whether we are resolving against a CSS @font-face
rule, and if so which one (e.g. applying descriptors from the @font-face
rule, deciding which @font-feature-values rules to apply, etc).
To achieve this we store the data required to resolve font features in a
struct and pass that to `FontComputer` which resolves the font features
and stores them with the computed `Font`.
We no longer need to invalidate the font shaping cache when features
change since the features are defined per font (and therefore won't ever
change).
Previously, SkTextBlob was built on every paint in
DisplayListPlayerSkia::draw_glyph_run(), which meant:
- Repeated work when the same display list is painted multiple times
- Glyph arrays were allocated and populated on each paint
Now the blob is built once during display list recording and cached in
GlyphRun.
ApplyTransform is no longer recorded to the display list. Transforms are
now applied inline during display list execution when switching between
accumulated visual contexts.
Change apply_transform to accept parameters directly instead of the
ApplyTransform struct.
Move the visual viewport (pinch-to-zoom) transform from a reserved slot
in DisplayList to the AccumulatedVisualContext tree as a root transform
node. Fixed position elements now correctly inherit from this context.
This requires rebuilding the context tree and display list on each zoom
change, but this overhead will be eliminated by future partial context
tree rebuilds.
Previously, both mask and clip-path were rendered to separate mutable
Gfx::Bitmap objects which forced CPU rasterization. They were then
combined using a CPU pixel-by-pixel operation before being returned
as an ImmutableBitmap.
Instead of including mask in the final bitmap as already rasterized
images, we now use display lists which opens opportunity to utilize
GPU if available.
Bitmap::apply_mask() and ApplyMaskBitmap display list command are no
longer used and have been removed.
Effects (opacity, blend mode, filters) must be applied in the parent's
coordinate space, before the element's transform. Previously this was
handled by manually switching to the parent's visual context when
applying effects at paint time.
By adding EffectsData to AccumulatedVisualContext and positioning it
before TransformData in the chain, effects are now naturally applied in
the correct order during display list replay, eliminating the special
case in StackingContext::paint().
For SVG filters that can generate content from empty elements (feFlood,
feImage, feTurbulence), a transparent FillRect command is emitted to
trigger the filter through the same AVC pipeline.
Previously, clip-path was applied only during painting in
StackingContext::paint(), which meant hit testing did not respect
clip-path boundaries. Clicks outside the visible clipped region but
inside the element's bounding box would incorrectly register as hits.
By moving clip-path into AccumulatedVisualContext, it becomes part of
the same system that handles transforms, clips, and scroll offsets for
both painting and hit testing, ensuring consistent behavior.
Remove the PushStackingContext and PopStackingContext display list
commands that are no longer used after the AccumulatedVisualContext
integration.
Previously, PushStackingContext was responsible for:
- Applying CSS transforms via its StackingContextTransform field
- Managing opacity layers
- Handling clip paths
- Tracking blend modes
All of this functionality has been replaced by:
- Transform/perspective tracking via AccumulatedVisualContext nodes
- The ApplyEffects command for opacity, blend modes, and filters
- The AddClipPath command for clip paths
Integrate AccumulatedVisualContext with display list recording and
playback. This is the main commit of the refactoring that delivers the
architectural improvements enabled by AccumulatedVisualContext.
Recording changes:
Each display list command now stores a single
RefPtr<AccumulatedVisualContext> instead of separate scroll_frame_id
and ClipFrame. The recorder simply captures the current accumulated
context when appending commands.
The before_paint()/after_paint() hooks that pushed/popped scroll frame
IDs are replaced by directly setting accumulated_visual_context on the
recorder before painting each element.
Playback changes:
The display list player now uses LCA (Lowest Common Ancestor) based
traversal to switch between visual contexts efficiently. When
transitioning from context A to context B:
1. Find the LCA of A and B in the context tree
2. Pop (restore) states back to the LCA depth
3. Push (save + apply) states from LCA down to B
This approach minimizes redundant save/restore operations. For example,
when rendering siblings that share a common scroll container, the
player keeps that scroll state applied and only switches the divergent
parts of their context chains.
Key deletions:
- Remove translate_by() from all 45 display list commands - commands
are now immutable
- Remove transform/perspective fields from PushStackingContext -
transforms are tracked via AccumulatedVisualContext
- Remove push_scroll_frame_id()/pop_scroll_frame_id() from
DisplayListRecorder
- Remove before_paint()/after_paint() hooks from Paintable
- Merge ApplyOpacity, ApplyCompositeAndBlendingOperator, ApplyFilter
into single ApplyEffects command
Stacking context painting changes:
The StackingContext::paint() method is significantly simplified.
Instead of building a PushStackingContextParams struct with transform
matrices and pushing/popping stacking contexts, it now:
1. Sets the accumulated visual context (which already contains
transforms)
2. Applies effects (opacity, blend mode, filters) if needed
3. Applies clip path if needed
4. Paints the content
5. Restores state
The visual state management that was interleaved throughout the
painting code is now handled uniformly by the context tree.
Introduce a new AddClipPath display list command that applies a
path-based clip region. This command is needed for the upcoming removal
of PushStackingContext, which currently handles clip paths as part of
its functionality. By extracting clip path handling into a dedicated
command, we can eliminate PushStackingContext entirely in favor of the
more granular AccumulatedVisualContext approach.
Adds pinch event handling that adjusts the VisualViewport scale and
offset. VisualViewport's (offset, scale) is then used to construct a
transformation matrix which is applied before display list execution.
This makes it so that the bounds for any paint-contained stacking
context are not derived from its children, but rather just set to
the rectangle that they will be clipped to anyways due to the paint
containment. Should make rendering faster on pages that use paint
containment.
Teach the display list executor to derive a bounding rectangle for
stacking contexts whose inner commands can all report bounds, that is,
most contexts without nested stacking contexts.
This yields a large performance improvement on https://tc39.es/ecma262/
where the display list contains thousands of groups like:
```
PushStackingContext blending=Multiply
DrawGlyphRun
PopStackingContext
```
Previously, `PushStackingContext` triggered an unbounded `saveLayer()`
even when the glyph run lies wholly outside the viewport. With this
change, we (1) cull stacking contexts that fall outside the viewport and
(2) provide bounds to `saveLayer()` when they are visible.
With this change rendering thread goes from 70% to 1% in profiles of
https://tc39.es/ecma262/. Also makes a huge performance difference on
Discord.
