Pass direct Core::EventLoop references through media producer and
playback plumbing that posts work back to the main thread. The browser
process main loops stay alive for the process lifetime, so these paths
no longer need weak event loop references.
Update LibMedia tests to pass their stack event loop directly to the
producer helpers.
This shouldn't actually be necessary, since the sample iterator will
resume from where it hit the error last anyway. Now that the decoded
data producers don't clear their queues when the demuxer doesn't move,
this works just fine for EOF.
This could easily be triggered by scrubbing aborting reads to restart
seeks, which would force us to decode from a prior keyframe instead of
continuing to decode from where the last seek left off if it's faster.
Instead of using a playback state to initiate and wake from suspension,
suspend the decoders themselves based on the time since the last status
check or pull. This allows audio and video to suspend independently,
and fixes videos getting stuck after suspension due to the waking seeks
being skipped downstream of the decoder, leaving the decoder suspended.
Seeks don't always move a decoded data producer's head, so we need to
make sure not to remove queued data downstream when that is the case.
To communicate this, the producers can now be queried before pulling
data, allowing them to have an in-band signal to clear the queued data
after a seek has moved the producer and broken monotonicity.
This fixes a flake in HTMLVideoElement-resize-event-during-playback.
Previously, we weren't too consistent about the definition of frame and
sample when it relates to raw audio data. This brings all the usages in
the context of raw data in line (hopefully), with samples referring to
a single PCM value, and frames referring to the multiple samples that
make up an instant's audio across all channels.
This is an intermediate step towards unifying the pipeline around new
Producer/Sink interfaces. Producers now have a pull() method that gets
the next piece of data from them. The pull() method returns a status
that can indicate whether it has current data, and if not, why it's
unavailable. This signal will be passed down the pipeline to the final
sink, which can expose the signal to its user, which in the normal
playback pipeline is PlaybackManager. The signal can be used to
transition between playback states. Currently, this is only hooked up
to the buffering state, but should be used later for ending playback
as well as decoding error propagation.
Buffering is now determined solely based on whether the pipeline is
blocked on incomplete data, so the ready state for video now progresses
past HAVE_METADATA immediately after playback manager initializes. This
will change when files have buffered ranges.
- Provider -> producer
- (Audio|Video)DataProvider -> Decoded(Audio|Video)Producer
- MediaTimeProvider remains suffixed Provider, moves out of the
Providers folder to the root of LibMedia
This brings the naming more in line with the intended split
functionality split between different nodes in the pipeline.
This commit splits out synchronization primitives from LibThreading into
LibSync. This is because LibThreading depends on LibCore, while LibCore
needs the synchronization primitives from LibThreading. This worked
while they were header only, but when I tried to add an implementation
file it ran into the circular dependency. To abstract away the pthread
implementation using cpp files is necessary so the synchronization
primitives were moved to a separate library.
Apparently this function uses a bitrate heuristic to determine which
track is best. We don't want or need that, so just select the first
track with default disposition (e.g. FlagDefault=1 in Matroska).
Most WebM files don't have their default duration defined, so we need
to parse the Opus frame header to determine the duration. This is
needed for buffered range calculation.
Instead of using a single track entry for all blocks in the file, use a
lookup to get the info needed to calculate the timestamp for the
specific track a block belongs to. No change in behavior for
SampleIterator, since that only returns blocks from the track that was
passed. This will be useful for MSE, since it demuxes all tracks at
once.
Otherwise, the promise handlers may never be invoked, and we won't know
whether we're matching expectations.
Prior to 39d865b, this test did not actually check if the PulseAudio
stream was able to connect before passing the test. After that commit,
it would instead try to post the rejection to the main thread, which
would either lead to a crash preventing a use-after-free on the main
event loop, or in the more common case, the event loop would be freed
and never invoke the rejection callback.
This allows us to avoid returning a PlaybackStream in cases where the
async initialization fails.
This is a step towards more graceful fallbacks when audio fails in
AudioMixingSink.
For web audio, I reckon an occasional misjudged channel layout is
better than more frequent exceptions.
Signed PCM is normalized with unsigned max divided by 2, not
signed max. If you divide by the signed max (32767), you get headroom
that can exceed the threshold below -1.0. It's not audible, this mostly
matters for tests that assume correct normalization. But it turns out
there's no shortage of "golden ears" jackholes out there who swear they
can hear the difference.
The way that other classes interact with IncrementallyPopulatedStream
is now through a virtual interface MediaStream and MediaStreamCursor.
This way, we can have simpler implementations of reading media data
that will not require an RB tree and synchronization.
...and abstract away the stream/cursor blocking/aborting functionality
so that demuxers can implement or ignore those methods as they see fit.
This is a step towards implementing a wrapper demuxer for MSE streams.
We were allowing Matroska blocks with fixed-size lacing to contain
frames with non-divisible sizes. This should not be possible, as it
inherently means that trailing bytes will be discarded.
We now have a valid and invalid testcase for fixed-size lacing to
ensure our handling remains correct.
We don't actually need a Vector stack of bytes read for each element
we're reading out of a Matroska file, we already have the C++ stack
in which we can store the start and end of the master elements we're
reading.
This fixes an issue where seeks while parsing master elements would not
increment m_octets_read, so the master element could continue reading
further than intended.
This could cause a BlockGroup followed by a SimpleBlock to read as if
the BlockGroup contained the SimpleBlock, meaning that SampleIterator
would skip the SimpleBlock.
A test is added to ensure this doesn't regress again.
This saves us from having our own color conversion code, which was
taking up a fair amount of time in VideoDataProvider. With this change,
we should be able to play high resolution videos without interruptions
on machines where the CPU can keep up with decoding.
In order to make this change, ImmutableBitmap is now able to be
constructed with YUV data instead of an RBG bitmap. It holds onto a
YUVData instance that stores the buffers of image data, since Skia
itself doesn't take ownership of them.
In order to support greater than 8 bits of color depth, we normalize
the 10- or 12-bit color values into a 16-bit range.
This fixes a compile issue on FreeBSD where this would not compile as
the `pulse/pulseaudio.h` header is not in the default search path,
instead it is in `/usr/local/include'. This is a problem because this
test manually includes `PulseAudioWrappers.h`.
This ensures that we're using the reader for the particular thread that
the block was read from, avoiding any race conditions between seeks and
reads across threads.
We only need to get the frames from a block when requested by the
demuxer, so factor that out into a function that it can call when it is
outputting frames.
Implement PlaybackStream using WASAPI. The design is similar to
PlaybackStreamAudioUnit in that it uses a task queue. A high priority
thread is used to render the stream. All the stream controls save for
the exit being requested which happens on destruction of the stream are
managed by the render thread.
Due to the design of the windows audio mixer the audio we receive must
be resampled to match the sample rate of the mixer. We use a float based
interleaved PCM stream which matches both our existing code and the
audio mixer which internally usues floats.
Having to use a mutex around a queue for the task queue is suboptimal,
in a future PR a MPSC queue could be added to AK and used instead.
Refactor the FFmpeg and Matroska demuxers to consume data through
`IncrementallyPopulatedStream::Cursor` instead of a pointer to fully
buffered.
This change establishes a new rule: each track must be initialized with
its own cursor. Data providers now explicitly create a per-track context
via `Demuxer::create_context_for_track(track, cursor)`, and own pointer
to that cursor. In the upcoming changes, holding the cursor in the
provider would allow to signal "cancel blocking reads" so an
in-flight seek can fail immediately when a newer seek request arrives.
