Divide a .wav file according to the video frame rate in Matlab

I have a yuv video (say stream.yuv) and the corresponding audio file (stream.wav). Now I have GUI which renders the raw video on a frame by frame basis. The problem I am currently facing is that I need to play the audio associated with each video frame.

I've tried the following to begin with, but when I execute this, the audio sounds chop

%% Dividing the audio into per-second samples
nframes = 720;
[audioFile, audioSampleFreq] = audioread('stream.wav');
numSamples = length(audioFile); 
audioLength = round(numSamples / audioSampleFreq);

for frame = 1:audioLength
    start = (frame-1)*audioSampleFreq+1;
    stop = frame*audioSampleFreq; 
    [start stop]
    audioFrame1 = audioFile(start:stop,:);
    sound(audioFrame1,audioSampleFreq);
end

To divide the audio into samples per video frame,

    audioFileName = 'stream.wav';
    nframes = 720;
    framerate = 25;
    [audioFile, audioSampleFreq] = audioread(audioFileName);

    audioFRate = round(audioSampleFreq/framerate);

    %% Total number of audio samples
    numSamples = length(audioFile); 

    %% number of audio frames
    numFrames = floor(numSamples/audioFRate);  

    for frame = 1:numFrames
        start = (frame-1)*audioFRate+1;
        stop = frame*audioFRate; 
        [start stop]
        audioFrame1 = audioFile(start:stop,:);
        sound(audioFrame1,audioSampleFreq);   
    end

Any thoughts on how to synchronize the per-frame rendering of the YUV frames and the per-second audio samples? Thanks!

Solved

I think you approached the problem the wrong way round. Adapting your audio signal to the video signal is typically the wrong way round because small disruptions in a audio stream are very noticeable, while the average viewer does not notice missing frames. Instead, synchronize your video signal to the audio signal. To get a quick start, set up a audio player and start with some code like this:

videorate=25 %fps
audiorate=audio.SampleRate %sample rate
while audio.isplaying()
   displayFrame(ceil(audio.CurrentSample/audiorate*videorate))
end

There is much which can be improved but I assume this simple implementation will already show better results than your current attempt.

Things to improve

• Don't render the same frame twice.
• The ceil is just a quick and dirty approximation. Instead you need to call your displayFrame for frame f at time f/fps-processing_delay where processing_delay is the time between calling displayFrame and the image actually appearing on the screen. Set it to 1/25 initially and tweak it manually if audio and video seems to be not synchronized.
• When missing the time where you should call displayFrame make a decision. For small times still display frame, for larger gaps drop the frame and continue with the next one. Maybe pause to display not to early.
• When your Video signal really messes up, e.g. dropping 15 out of the last 25 frames, check for possible causes. Processing high resolution raw video, this typically happens because your HDD is to slow, so pausing both video and audio for a short time should allow the buffers to fill up again.

An alternative to this approach are approaches using variable playback speed, slightly adjusting the audio and video playback speed to get both synchronized. I assume that such approaches won't succeed in Matlab because you need to process your data with very low latencies to get this approach done.