If a set_time()'s callbacks were still pending, but a resume() went
through, the audio timing could get a bit confused. Instead, check if
m_temporary_time is set before resuming, since that would indicate that
the set_time() callbacks will call resume() when the time is stable
again.
Also, multiple set_time() calls will now only start one set of tasks,
and m_temporary_time is used to store the time passed to the last call.
Thus, set_time() doesn't conflict with itself, but no calls to
set_time() are ignored still.