We could potentially have a pull() that would output audio after a
status query said that data was pending. Instead, ensure that we keep
the subsequent pull() from providing anything in such cases.
Also, rewrite HaveData to Pending when it outputs nothing. When
multiple audio tracks were enabled, it was possible to reach the empty
block branch with a HaveData status, since it takes priority over
Pending. This fixes a crash in pull() verifying that the combined
status is not HaveData.
Seeks don't always move a decoded data producer's head, so we need to
make sure not to remove queued data downstream when that is the case.
To communicate this, the producers can now be queried before pulling
data, allowing them to have an in-band signal to clear the queued data
after a seek has moved the producer and broken monotonicity.
This fixes a flake in HTMLVideoElement-resize-event-during-playback.
This isn't strictly an optimization, but at this stage that is its
effect. By not signaling unless the combined state has changed, we
avoid waking the downstream node unnecessarily. However, the real
purpose is to allow us to transmit a new signal when the upstream
position moves to clear the data downstream to allow repeat timestamps.
Previously, we weren't too consistent about the definition of frame and
sample when it relates to raw audio data. This brings all the usages in
the context of raw data in line (hopefully), with samples referring to
a single PCM value, and frames referring to the multiple samples that
make up an instant's audio across all channels.
Now, all nodes are connected through Sink::connect_input() and
disconnect_input().
AudioMixer now derives from a base AudioProcessor class that inherits
from both AudioSink and AudioProducer. It is the only current node that
can accept multiple inputs, tracking each one by its pointer identity.
Seeking is now unified under one single method signature implemented by
all producers and transmitted through the pipeline by all sinks. By
doing it this way, we can simply instantaneously notify each node of
the pipeline that it needs to stop what it's doing and seek. For nodes
that are threaded (particularly the source providers), this causes them
to stop pushing data to their queue immediately, so that no stale data
makes it through to the output. Then, when new data does come through,
that is a clear indication that the seek has completed.
Note that track enablement is now through the pipeline as well, which
means that SuspendedStateHandler no longer has a way to suspend newly-
enabled tracks. Decoder suspension will need to be reworked to fit into
this new pipeline, sleeping/disposing and restarting entirely based on
the pull() timing in the producers.
Using a callback shared by all producers in the pipeline, notify the
AudioPlaybackSink when it needs to wake up and start processing data
again. Prior to this commit, it was simply burning CPU spinning until
data was produced.
This is an intermediate step towards unifying the pipeline around new
Producer/Sink interfaces. Producers now have a pull() method that gets
the next piece of data from them. The pull() method returns a status
that can indicate whether it has current data, and if not, why it's
unavailable. This signal will be passed down the pipeline to the final
sink, which can expose the signal to its user, which in the normal
playback pipeline is PlaybackManager. The signal can be used to
transition between playback states. Currently, this is only hooked up
to the buffering state, but should be used later for ending playback
as well as decoding error propagation.
Buffering is now determined solely based on whether the pipeline is
blocked on incomplete data, so the ready state for video now progresses
past HAVE_METADATA immediately after playback manager initializes. This
will change when files have buffered ranges.
This will allow reuse of the allocations, instead of reallocating a
FixedArray for every block that changes size. Generally, it will be
possible to reuse AudioBlock memory throughout most of the pipeline
at least while in a single process.
- Provider -> producer
- (Audio|Video)DataProvider -> Decoded(Audio|Video)Producer
- MediaTimeProvider remains suffixed Provider, moves out of the
Providers folder to the root of LibMedia
This brings the naming more in line with the intended split
functionality split between different nodes in the pipeline.
This doesn't actually change things too much from the prior commit, but
acts as a step towards making mixing into a sink/provider combo in the
new pipeline model.
