Return pts of first frame in audio API #552
Conversation
facebook-github-bot
added
the
CLA Signed
| @@ -838,7 +838,7 @@ VideoDecoder::FrameBatchOutput VideoDecoder::getFramesPlayedInRange( | |||
| return frameBatchOutput; | |||
| } | |||
|
|
|||
| torch::Tensor VideoDecoder::getFramesPlayedInRangeAudio( | |||
| VideoDecoder::AudioFramesOutput VideoDecoder::getFramesPlayedInRangeAudio( | |||
There was a problem hiding this comment.
I decided to create a new AudioFramesOutput struct instead of relying on the existing FrameOutput, which contains unnecessary fields like streamIndex and durationSeconds. For now, those aren't needed for audio. No super strong opinion though.
There was a problem hiding this comment.
I think that's the right call, since the tensors are going to be very different.
| tensors.push_back(frameOutput.data); | ||
| firstFramePtsSeconds = | ||
| std::min(firstFramePtsSeconds, frameOutput.ptsSeconds); | ||
| frames.push_back(frameOutput.data); |
There was a problem hiding this comment.
It's now a bit ugly that we are manipulating both a FrameOutput and an AudioFramesOutput here in this function.
This is mostly because convertAVFrameToFrameOutput returns a FrameOutput, but maybe we could bypass it and directly call convertAudioAVFrameToFrameOutputOnCPU. I'll write a TODO to investigate this.
There was a problem hiding this comment.
Oh, yeah, that's cumbersome. If we're in an audio-only call, we shouldn't have to deal with video structures.
I thought we didn't need this, but we actually need it in order to implement the public sample-based API (which is out of scope for this PR).