Quicktime: video and sound drifting apart

brian · October 26, 2022, 1:56am

I agree. Variable frame rate or media that does not seek well is the most common cause for this symptom. The conversion should fix either of those.

david.lyon · October 26, 2022, 2:09am

shotcut · October 26, 2022, 2:50am

When many of us here say “convert” we mean the Convert button in Properties

Austin · October 26, 2022, 4:22am

To make life extra complicated, -r might still create a variable frame rate video with some output formats. Matroska in particular is VFR by default. To guarantee a constant frame rate output, -vsync cfr has to be specified with -r (or use the fps filter instead), which is often paired with -filter:a aresample=async=1:min_comp=0.001:min_hard_comp=0.1:first_pts=0 to match the audio with the video whenever retiming happens. A quick overview here:

https://trac.ffmpeg.org/wiki/ChangingFrameRate

The -fflags +genpts -r 30 method can be dangerous because it is only rewriting the presentation timestamps (PTSes) without modifying the frame sequence itself. If the incoming video is VFR where one second has 22 frames and the next second has 30 frames, then +genpts will blindly write new PTSes with the same 1/30th time interval per frame, which in this case would make the video appear to suddenly speed up because it moved 8 frames from the 2nd second into the 1st second to meet the 30-frame quota. By using -vsync cfr -r 30 or -filter:v fps=30, frames will be duplicated or dropped as necessary to retain their expected timing. Although, +genpts is still a cool tool for doing lossless retiming of video (CFR source assumed):

https://trac.ffmpeg.org/wiki/How%20to%20speed%20up%20/%20slow%20down%20a%20video

For anyone curious, the ffmpeg command built by Shotcut for converting videos to an edit-friendly format can be found in the source code here:

github.com

mltframework/shotcut/blob/3deec9997c3f0ff4b6b107237bbe7b97d0228957/src/widgets/avformatproducerwidget.cpp#L811

      
        
                    basename = QString();
                }
                for (unsigned i = 1; i < std::numeric_limits<unsigned>::max(); i++) {
                    QString filename = QString::fromLatin1("%1%2.%3").arg(basename).arg(i).arg(extension);
                    if (!info.dir().exists(filename))
                        return info.dir().filePath(filename);
                }
                return filePath;
            }
            
            
void AvformatProducerWidget::convert(TranscodeDialog &dialog)
            {
                int result = dialog.exec();
                if (dialog.isCheckBoxChecked()) {
                    Settings.setShowConvertClipDialog(false);
                }
                if (result == QDialog::Accepted) {
                    QString resource = Util::GetFilenameFromProducer(producer());
                    QString path = Settings.savePath();
                    QStringList args;
                    QString nameFilter;

david.lyon · October 26, 2022, 2:58pm

They mean ‘dangerous’ in a very comedic sense. It’s not like Film negatives will get burnt. Digital Editing is non-destructive. You would still have your source video files somewhere like on your camera. Maybe these aren’t your words

Whilst I don’t have any Concerts to go to. I could film something else, like me watching youtube or a 3D print and see what happens here. I need an excuse to clean up and that camera is begging for work otherwise it has to go in the bin.

Generally speaking as far as my point of view is concerned, the issue is in the Menu-settings. If the debate is around having the the “-fflags +genpts -r 30” on or off that should probably left to the user as settings for the Quicktime Import.

There don’t appear to be any settings that I could find for the video input. That may or may not be ok. There is a “Synchronisation” option and it could go there I guess.

In the end, I stopped shooting with Quicktime on Windows with that old camera. It’s too problematic and changed the setting on my camera to shoot in mpeg. I then also realised it was time to buy a new Panasonic.

The point is that Quicktime is no longer current and it’s fallen by the wayside I would say because these days generally “everyone wants to shoot in 4K RAW” which they don’t but anyway that’s what they say.

The question is “is it even worth supporting nicely?” Having ffmpeg options for different import formats seems somewhat sensible and flexible.

I’ll record some Quicktime tomorow and find out just how bad this problem is.

Frank183 · October 26, 2022, 4:16pm

So, I finally found some time again for this. I did the conversion as suggested, and extracted one song from the end of the converted video and now video and sound are perfectly aligned. So, it looks like that the audio enhanced MOV file has some technical issues with frame rate, time stamps or the like.

Thank you so much everybody, I was getting real desperate!!!

Now I have to figure out which quality settings to use. When I keep the slider all the way to the left then the original 2GB MOV file turns into a 5 GB MP4 file, if I put the slider into the middle it becomes 36 GB. I can’t see with my bare old eyes any differences between those, so I guess the default settings will do.

Now, while I’m at it, I’m shamelessly asking for more advice, if it is ok. SC is great and very intuitive, but 2 things I haven’t found out yet, maybe somebody can point me to the right section of the documentation:

I’m using these filters: zoom/position, brightness, contrast, fade in/fade out. I am editing the concert song by song and I would love to use the very same filter combination for every song. How can I do this?

I would love to edit the concert song by song, but only somehow save the settings and then let the final output files render overnight, batch processing so to speak.

If anybody has a link for either one of those at hand, I’d be very gratetul.

Thank you so much again and I will share the final result.

Best regards
Frank

Elusien · October 26, 2022, 5:05pm

You can save the various filter options as a Preset and then apply that filter with that preset.

You can copy all the filters to the clipboard and then paste them as a set to another clip.

Here the Preset will save the Brightness options and the Copy button will copy both the Brightness and Color-Grading filters to the clipboard to be pasted to another clip with the Paste button.

Austin · October 26, 2022, 5:08pm

Open a project and start the export. Without closing Shotcut, open another project and export it too. The Jobs panel will show the second export in a queue. Load up the queue as much as you want, then get some sleep.

david.lyon · October 27, 2022, 4:09pm

This is how I would use nested ,MLTs for every song then combine them at the end in a ‘full-concert.mlt’.

That way you can have individual filters for each song and global filters for the whole song.

I believe this method would allow you to completely contain the Audio drift.

Frank183 · November 1, 2022, 10:58am

Hi everyone,

sorry for not getting back earlier, life is keeping me real busy at the moment.
Thank you so much for all the great advice and help!!!

Here’s the output, enjoy! (I’m the sax player on the far right)

Best regards
Frank

Bridget_Murphy · November 27, 2023, 3:55am

Aligning video and sound tracks manually, especially with music, is challenging due to the precision needed, particularly for visual cues like a drummer hitting a drum; consider using specialized video editing software for more precise control or exploring online resources for tips on improving audio and video synchronization in your project.

Bridget_Murphy · December 1, 2023, 2:44am

That’s a fascinating insight into the timing dynamics during a live performance. Adjusting for the speed of sound and syncing the audio and video makes a lot of sense, especially when you want to capture those drum hits precisely. I’ll definitely give the calculation and adjustment a try, considering the distance from the stage and the frame rate.

Bridget_Murphy · December 5, 2023, 11:48am

Correct me if I am wrong but altitude also make an impact on sound speed?