Previously, we weren't too consistent about the definition of frame and
sample when it relates to raw audio data. This brings all the usages in
the context of raw data in line (hopefully), with samples referring to
a single PCM value, and frames referring to the multiple samples that
make up an instant's audio across all channels.
This is an intermediate step towards unifying the pipeline around new
Producer/Sink interfaces. Producers now have a pull() method that gets
the next piece of data from them. The pull() method returns a status
that can indicate whether it has current data, and if not, why it's
unavailable. This signal will be passed down the pipeline to the final
sink, which can expose the signal to its user, which in the normal
playback pipeline is PlaybackManager. The signal can be used to
transition between playback states. Currently, this is only hooked up
to the buffering state, but should be used later for ending playback
as well as decoding error propagation.
Buffering is now determined solely based on whether the pipeline is
blocked on incomplete data, so the ready state for video now progresses
past HAVE_METADATA immediately after playback manager initializes. This
will change when files have buffered ranges.
This will allow reuse of the allocations, instead of reallocating a
FixedArray for every block that changes size. Generally, it will be
possible to reuse AudioBlock memory throughout most of the pipeline
at least while in a single process.
While mixing doesn't cost much time at the moment, moving audio
processing off the callback thread means that the callback itself can
block for a minimal amount of time regardless of the amount of time
needed to actually process the audio data.
This matters a lot for AudioUnit, where the entire daemon can start to
glitch if delays occur.
The goal of this change is to split mixing off into a separate class,
and process audio on both ends of the mixer synchronously. This should
allow a more declarative approach to processing audio, where inserting
a time stretching processor, for example, becomes much simpler.
In this commit, the thread spin-waits for new upstream data. Making it
wait on a condition variable here would overcomplicate this commit, so
it's deferred until a later commit establishes that forward wakeup.
Audio blocks now contain a sample specification with the sample rate
and channel map for the audio data they contain. This will facilitate
conversion from one sample specification to another in order to allow
playback on devices with more or less speakers than the audio data
contains.
This prevents PlaybackManager's seek while enabling an audio track from
causing the AudioMixingSink to push audio blocks forward unnecessarily.
Previously, the seek would cause the initial block or blocks to repeat
from the perspective of AudioMixingSink, so it would think that it
needs to shift the first block after the seek forward by a few samples.
By moving this to the AudioDataProvider, we can clear the last sample
index every time the decoder is flushed, ensuring that the block
shifting always makes sense.
By doing this in AudioMixingSink instead of the Decoder
implementations, we avoid having to duplicate this shifting logic
across multiple implementations.
This also fixes an issue where multiple audio blocks occupying the same
timestamp would be skipped while seeking, causing a significant break
in audio.
This commit implements the functionality to play back audio through
PlaybackManager.
To decode the audio data, AudioDataProviders are created for each track
in the provided media data. These providers will fill their audio block
queue, then sit idle until their corresponding tracks are enabled.
In order to output the audio, one AudioMixingSink is created which
manages a PlaybackStream which requests audio blocks from multiple
AudioDataProviders and mixes them into one buffer with sample-perfect
precision.
