dmaax/ladybird
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
ladybird/Libraries/LibMedia/VideoFrame.cpp

32 lines
612 B
C++
Raw Normal View History

LibVideo: Add VideoFrame class for decoded video frames The class is virtual and has one subclass, SubsampledYUVFrame, which is used by the VP9 decoder to return a single frame. The output_to_bitmap(Bitmap&) function can be used to set pixels on an existing bitmap of the correct size to the RGB values that should be displayed. The to_bitmap() function will allocate a new bitmap and fill it using output_to_bitmap. This new class also implements bilinear scaling of the subsampled U and V planes so that subsampled videos' colors will appear smoother.
2022-10-08 20:36:57 -03:00
/*
LibGfx+LibMedia: Send video frames to Skia as subsampled YUV This saves us from having our own color conversion code, which was taking up a fair amount of time in VideoDataProvider. With this change, we should be able to play high resolution videos without interruptions on machines where the CPU can keep up with decoding. In order to make this change, ImmutableBitmap is now able to be constructed with YUV data instead of an RBG bitmap. It holds onto a YUVData instance that stores the buffers of image data, since Skia itself doesn't take ownership of them. In order to support greater than 8 bits of color depth, we normalize the 10- or 12-bit color values into a 16-bit range.
2026-01-20 23:10:33 -03:00
* Copyright (c) 2022-2026, Gregory Bertilson <gregory@ladybird.org>
LibVideo: Add VideoFrame class for decoded video frames The class is virtual and has one subclass, SubsampledYUVFrame, which is used by the VP9 decoder to return a single frame. The output_to_bitmap(Bitmap&) function can be used to set pixels on an existing bitmap of the correct size to the RGB values that should be displayed. The to_bitmap() function will allocate a new bitmap and fill it using output_to_bitmap. This new class also implements bilinear scaling of the subsampled U and V planes so that subsampled videos' colors will appear smoother.
2022-10-08 20:36:57 -03:00
*
* SPDX-License-Identifier: BSD-2-Clause
*/
LibMedia+LibWeb: Store decoded YUV data in video frames Decoded video frames should own their planar YUV data and color space directly. Keeping that storage behind ImmutableBitmap gave a still-image abstraction media-specific behavior and made calls like bitmap() potentially allocate and convert a whole video frame. Move YUV ownership into Media::VideoFrame, where the lifetime naturally follows media playback, and remove the YUV-backed mode from ImmutableBitmap. This commit intentionally keeps the visible Web paint path on ExternalContentSource by converting the current frame back to an ImmutableBitmap where Web still expects one. Callers that need pixels now ask the frame to convert explicitly. That preserves behavior for canvas and bitmap consumers while making the expensive YUV-to-pixel path visible at the call site instead of hiding it behind ImmutableBitmap::bitmap().
2026-05-05 06:38:50 -03:00
#include <LibGfx/YUVData.h>
LibVideo: Add VideoFrame class for decoded video frames The class is virtual and has one subclass, SubsampledYUVFrame, which is used by the VP9 decoder to return a single frame. The output_to_bitmap(Bitmap&) function can be used to set pixels on an existing bitmap of the correct size to the RGB values that should be displayed. The to_bitmap() function will allocate a new bitmap and fill it using output_to_bitmap. This new class also implements bilinear scaling of the subsampled U and V planes so that subsampled videos' colors will appear smoother.
2022-10-08 20:36:57 -03:00
#include "VideoFrame.h"
LibMedia: Rename LibVideo to LibMedia This change is in preparation for implementing audio codecs into the library and using audio as timing for video playback.
2024-06-10 17:04:03 -03:00
namespace Media {
LibVideo: Add VideoFrame class for decoded video frames The class is virtual and has one subclass, SubsampledYUVFrame, which is used by the VP9 decoder to return a single frame. The output_to_bitmap(Bitmap&) function can be used to set pixels on an existing bitmap of the correct size to the RGB values that should be displayed. The to_bitmap() function will allocate a new bitmap and fill it using output_to_bitmap. This new class also implements bilinear scaling of the subsampled U and V planes so that subsampled videos' colors will appear smoother.
2022-10-08 20:36:57 -03:00
Everywhere: Run clang-format The following command was used to clang-format these files: clang-format-21 -i $(find . \ -not \( -path "./\.*" -prune \) \ -not \( -path "./Build/*" -prune \) \ -not \( -path "./Toolchain/*" -prune \) \ -type f -name "*.cpp" -o -name "*.mm" -o -name "*.h")
2026-02-17 21:50:23 -03:00
VideoFrame::VideoFrame(
AK::Duration timestamp,
AK::Duration duration,
LibVideo: Convert subsampled frames in a vectorization-friendly way This change reduces the time spent converting frames for display in a 1080p video from ~19ms per frame to ~12ms per frame.
2023-04-16 09:43:31 -03:00
Gfx::Size<u32> size,
LibMedia+LibWeb: Store decoded YUV data in video frames Decoded video frames should own their planar YUV data and color space directly. Keeping that storage behind ImmutableBitmap gave a still-image abstraction media-specific behavior and made calls like bitmap() potentially allocate and convert a whole video frame. Move YUV ownership into Media::VideoFrame, where the lifetime naturally follows media playback, and remove the YUV-backed mode from ImmutableBitmap. This commit intentionally keeps the visible Web paint path on ExternalContentSource by converting the current frame back to an ImmutableBitmap where Web still expects one. Callers that need pixels now ask the frame to convert explicitly. That preserves behavior for canvas and bitmap consumers while making the expensive YUV-to-pixel path visible at the call site instead of hiding it behind ImmutableBitmap::bitmap().
2026-05-05 06:38:50 -03:00
u8 bit_depth,
Gfx::ColorSpace color_space,
NonnullOwnPtr<Gfx::YUVData> yuv_data)
LibGfx+LibMedia: Send video frames to Skia as subsampled YUV This saves us from having our own color conversion code, which was taking up a fair amount of time in VideoDataProvider. With this change, we should be able to play high resolution videos without interruptions on machines where the CPU can keep up with decoding. In order to make this change, ImmutableBitmap is now able to be constructed with YUV data instead of an RBG bitmap. It holds onto a YUVData instance that stores the buffers of image data, since Skia itself doesn't take ownership of them. In order to support greater than 8 bits of color depth, we normalize the 10- or 12-bit color values into a 16-bit range.
2026-01-20 23:10:33 -03:00
: m_timestamp(timestamp)
, m_duration(duration)
, m_size(size)
, m_bit_depth(bit_depth)
LibMedia+LibWeb: Store decoded YUV data in video frames Decoded video frames should own their planar YUV data and color space directly. Keeping that storage behind ImmutableBitmap gave a still-image abstraction media-specific behavior and made calls like bitmap() potentially allocate and convert a whole video frame. Move YUV ownership into Media::VideoFrame, where the lifetime naturally follows media playback, and remove the YUV-backed mode from ImmutableBitmap. This commit intentionally keeps the visible Web paint path on ExternalContentSource by converting the current frame back to an ImmutableBitmap where Web still expects one. Callers that need pixels now ask the frame to convert explicitly. That preserves behavior for canvas and bitmap consumers while making the expensive YUV-to-pixel path visible at the call site instead of hiding it behind ImmutableBitmap::bitmap().
2026-05-05 06:38:50 -03:00
, m_color_space(move(color_space))
, m_yuv_data(move(yuv_data))
LibVideo: Convert subsampled frames in a vectorization-friendly way This change reduces the time spent converting frames for display in a 1080p video from ~19ms per frame to ~12ms per frame.
2023-04-16 09:43:31 -03:00
{
LibMedia: Store YUV planes as byte arrays with no padding for 8-bit This should halve the size of frames in memory for frames with 8-bit color components, which is the majority of videos. Calculation of the size of subsampled planes has also been consolidated into a struct. There are likely some places that will still need to change over to this, but it should prevent issues due to differing handling of rounding/ceiling.
2024-06-20 23:50:54 -03:00
}
LibGfx+LibMedia: Send video frames to Skia as subsampled YUV This saves us from having our own color conversion code, which was taking up a fair amount of time in VideoDataProvider. With this change, we should be able to play high resolution videos without interruptions on machines where the CPU can keep up with decoding. In order to make this change, ImmutableBitmap is now able to be constructed with YUV data instead of an RBG bitmap. It holds onto a YUVData instance that stores the buffers of image data, since Skia itself doesn't take ownership of them. In order to support greater than 8 bits of color depth, we normalize the 10- or 12-bit color values into a 16-bit range.
2026-01-20 23:10:33 -03:00
VideoFrame::~VideoFrame() = default;
LibVideo: Fast-path converting colors by only matrix coefficients We don't need to run through the whole floating-point color converter for videos that use sRGB transfer characteristics and BT.709 color primaries. This commit adds a new templated inlining function to ColorConverter to do a very fast fixed-point YCbCr to RGB conversion. With the fast path, frame conversion times go from ~7.8ms down to ~3.7ms. The fast path can benefit a lot more from extra SIMD vector width, as well.
2023-04-20 07:11:57 -03:00
LibVideo: Add VideoFrame class for decoded video frames The class is virtual and has one subclass, SubsampledYUVFrame, which is used by the VP9 decoder to return a single frame. The output_to_bitmap(Bitmap&) function can be used to set pixels on an existing bitmap of the correct size to the RGB values that should be displayed. The to_bitmap() function will allocate a new bitmap and fill it using output_to_bitmap. This new class also implements bilinear scaling of the subsampled U and V planes so that subsampled videos' colors will appear smoother.
2022-10-08 20:36:57 -03:00
}
Reference in a new issue Copy permalink