HowTo sync video clips to beat in music files

Update to rev 57:
There were detached audio clips from video clips missing in my calculation of the starttime of the beat markers. Fixed that in this revision. So now the list to select the music (to apply the beat markers to) is longer, as it does not only contain .mp3 and .wav files as before, but every clip on the timeline.

This is a great tool. There is still some manual work to do, but it makes it a LOT easier and faster to create beat markers in Shotcut.

And to make the beats easier to see in Audacity, I use the Equalization filter on the song and adjust the settings until the waveform shows almost only the beats in the waveform.

Here’s an example of a song’s waveform before and after equalization

Using Equalization helps too if you decide to use Audacity’s Beat finder (beat detection) tool. It will generate a lot less undesired labels, so you’ll have less to delete.

See the difference

2 Likes

There are much better plugins to do the job in audacity to find beats and give numbers to the rythm.
Search for VAMP-beat-root, it’s free.
I use “Bar and Beat Tracker: Beat Count” for the job.

4 Likes

@MusicalBox and @smartintech, this is incredibly useful information for me. Thank you.

2 Likes

You’re welcome. I like your tutorials on YT, they inspired me.

Rev 58: just added a button to minimize to the title bar.

Future plan: if anyone knows an easy way how to mod-script-pipe audacity to automatically analyze a given sound file and export the resulting text file, i will add that command line to the tool for every music clip on shotcuts timeline that has no beat label file already…

The versatility of the video editing tools sync video to music beat apps offer should help you determine if a video editor can cater to all your video editing needs. Thank you for sharing.

@smartintech Thanks for all the useful information that you shared on this topic. I want to use Shotcut and Audacity for creating beat-sync effects but I’m on Linux so I can’t use your exe. Could you tell me what is the algorithm in your application, please? Maybe I can create a Python script to handle this as a cross-platform solution.

Now I created my labels on Audacity but how do you convert timestamp from millisecond format to fps format for Shotcut?
Now my label text content is a thing like this:

0.231050	0.231050	B
0.600731	0.600731	B
1.181657	1.181657	B
....

Use the Timer filter as a reference.
The Timer filter is bound to the Video Mode fps.

Maybe you do not need to. If you are authoring MLT XML and seeking by typing into the timecode field of Shotcut read the bottom of the keyboard reference for how to enter time values: Shotcut - Keyboard Shortcut Reference

As far as i understood this, the clips are appended on the timeline. They do not have a start-time - just a length. So the gaps are filled with blank “clips”.
To insert something on a specific point in time you need to sum up the length of all the clips before that and create a blank clip in front of your “beat marker”.
Here you must pay attention to the timing conversion: i first read out the FPS of the project as a fraction and then convert between frames and time to set the right length.

So that’s practically my algorithm :wink:

Maybe there are better ways to do that. Maybe the MLT format has changed and allows for absolute timestamps. Maybe Shotcut will support markers that one can import from a file.

I can suggest using wine under linux, as it will run most VCPP static linked builds from me.

1 Like

Hello Martin, I wanted to use your solution but it does not seem to be working, I always get Read 0 producers.
I tried everything that could come up with like: sending files to the shorter filepath, using only latin letters with no spaces, having both file and _beat.txt files in the same location, reducing framerate, etc., but nothing worked

Could you, please, advise if there are any solutions to fix that?

If there are any known issues?

1 Like

I think you already did what i have had checked first. And my tool reads the framerate of your project correctly.
Even if my tool wouldn’t detect audio clips it would count all producers. But it found none.

So i assume the .mlt file format has changed as announced a while ago. (Maybe because of time remapping feature).
It’s likely that i have to adapt to that .mlt format change.

As i want to cut some videos in the next weeks i will either adapt my tool to the most recent shotcut version OR i will switch to davinci ressolve, as it offers motion tracking and speed ramping without a hassle and even color correction with my x-rite color checker.

I try to post an update about that here next week.

1 Like

Hello Martin, thanks.

Actually, I took your approach and figured out the current format of mlt file, so I created google spreadsheets file, that converts audacity file (3 columns) into xml-like format to paste into mlt file via notepad =D

Your explanation did help me a lot, I just had to use formulas to convert “absolute” seconds into “relative” (difference between ending of the last beat-shot to the start of the new one) seconds, also keeping in mind that i have to convert 1000ms into 60frames/sec, so I calculate “the exact second” from the left, to fit the exact moment, when shot should’ve started. Like, on my screenshot below, the audacity detected 28.195 would be between frames, so I use 28.183 that would be exact 00:28.11 (28sec 11 frames).
So I only have to cope and paste it inside xml (also delete if beat is more frequent that the length of a dummy shot =D), but that’s still convenient.

Again, thank you very much for your help, if I were better at programming, I might have created something similar, but since I am good at office tools and it is faster, I used excel/sheets file =D

1 Like

Great job. I like all types of solutions.

So i just had a look at the new MLT file and it has a new format. At a first glance the tag seems to be replaced by a tag.
I will dig deeper into that.

@smartintech, the latest version added timeline markers. If clip markers get added, could it be combined with your script to make a feature in Shotcut that marks the beats of a clip with audio?

1 Like

Yeah, i just saw the markers when trying the most recent version 21.10.31.
I like the idea.

But think about it, if it will be really useful, because i delete the extra track with all the markers after aligning my cuts to the beats. There are a lot of markers. And i can re-generate them any time.

So here is a quick and rarely tested solution that works for the most recent Shotcut 21.10.31:
It should state “Revision 58” in the title bar and is only intended for .MLT files created with newer versions of Shotcut.

As i found out that the new MLT format is only used on freshly generated projects, i hadn’t have the time today to build some more new test projects. And all my old test files remain almost the same when i load them into the new Shotcut version, change a bit and save again.

So feel free to test it and let me know about any issues so that i can make this quick shot a stable tool.

I kept the old link to the old tool as it still works if you load your old project into a new shotcut version.

1 Like

May i ask a question to the developers:
Is there any changelog describing the MTL format changes and how they are handeled?
For the first attempt of my tool here i used the producer tag in MLT ver. 6.21.0.
And in the most recent MLT ver. 7.1.0 i used the chain tag.
But when i open an old 6.21.0 MLT with the new Shotcut, edit something, append new clips and save again into a new file, this way it uses producer tags instead of chain tags - even though it writes the MLT version 7.1.0 into the header…

I just want to know which way to go to make this tool usable for the mayority of users here:

  1. Build two versions of my tool. One for the old producer tags, one for the new chain tags.
  2. Build one version, reading out the MLT version and then using producer OR chain tags.
  3. Build one version just using whatever tag is in the file (producer AND/OR chain).

And is it okay to generate the dummy clips for each audio beat as producers or do i have to switch to chains as well for the upcoming MLT versions?

Here is the new version of my tool to generate dummy clips for audio beats to align cuts to.

This is Revision 59 and it can deal with the old shotcut format as well as with the current one.
Please let me know if there are any issues with it.