HowTo sync video clips to beat in music files

Pretty cool workaround. Thanks for sharing!

1 Like

I found a misbehaviour of transitions when inserting my beat track between background and lowest video track. Shotcut even changed a lot of entries after saving my generated file.
So i changed this and put the beat track on top, making it “not show”.
When i compare the diff of the original MLT file and my generated one after removing the beat track in shotcut, there is no difference anymore. So i consider my tool to work properly.

Here is the new version 51 (i removed the link destination to the previous ver. 50)

AudioLabel2Shotcut.exe

This topic was automatically closed after 90 days. New replies are no longer allowed.

I did some improvments to the tool.
Link on google drive shall remain the same (i used overwrite). So have a look on the revision number.
Revision 56 is now the current one and is working fine on my projects.

Changelog:

Rev 56:
Fixed markers to be only applied within the in and out position of the audio clip (no matter what in/out times of that audio)
Rev 55:
Beat markers will not be added to the marker track further than the outtime of the music clip.
Rev 54:
Changed the generated outfilename to append a “_withMarkers” to the .mlt file.
Also added increased playlist ids to allow multiple beat marker tracks for more than one song at once in a project after using this tool multiple times.
Rev 53:
Fixed memory allocation of a buffer that made it crash on long filenames.
Rev 52:
Can apply markers also when audio clip does not start from the very beginning.

I described how it works in the video at 0:39

Link to the tool on my G drive is the same as before:

Let me know, if you have any issues with it.
I hope this is useful to someone and i want to say thankyou for all your contributions here.

1 Like

Update to rev 57:
There were detached audio clips from video clips missing in my calculation of the starttime of the beat markers. Fixed that in this revision. So now the list to select the music (to apply the beat markers to) is longer, as it does not only contain .mp3 and .wav files as before, but every clip on the timeline.

This is a great tool. There is still some manual work to do, but it makes it a LOT easier and faster to create beat markers in Shotcut.

And to make the beats easier to see in Audacity, I use the Equalization filter on the song and adjust the settings until the waveform shows almost only the beats in the waveform.

Here’s an example of a song’s waveform before and after equalization

Using Equalization helps too if you decide to use Audacity’s Beat finder (beat detection) tool. It will generate a lot less undesired labels, so you’ll have less to delete.

See the difference

2 Likes

There are much better plugins to do the job in audacity to find beats and give numbers to the rythm.
Search for VAMP-beat-root, it’s free.
I use “Bar and Beat Tracker: Beat Count” for the job.

4 Likes

@MusicalBox and @smartintech, this is incredibly useful information for me. Thank you.

2 Likes

You’re welcome. I like your tutorials on YT, they inspired me.

Rev 58: just added a button to minimize to the title bar.

Future plan: if anyone knows an easy way how to mod-script-pipe audacity to automatically analyze a given sound file and export the resulting text file, i will add that command line to the tool for every music clip on shotcuts timeline that has no beat label file already…

The versatility of the video editing tools sync video to music beat apps offer should help you determine if a video editor can cater to all your video editing needs. Thank you for sharing.

@smartintech Thanks for all the useful information that you shared on this topic. I want to use Shotcut and Audacity for creating beat-sync effects but I’m on Linux so I can’t use your exe. Could you tell me what is the algorithm in your application, please? Maybe I can create a Python script to handle this as a cross-platform solution.

Now I created my labels on Audacity but how do you convert timestamp from millisecond format to fps format for Shotcut?
Now my label text content is a thing like this:

0.231050	0.231050	B
0.600731	0.600731	B
1.181657	1.181657	B
....

Use the Timer filter as a reference.
The Timer filter is bound to the Video Mode fps.

Maybe you do not need to. If you are authoring MLT XML and seeking by typing into the timecode field of Shotcut read the bottom of the keyboard reference for how to enter time values: Shotcut - Keyboard Shortcut Reference

As far as i understood this, the clips are appended on the timeline. They do not have a start-time - just a length. So the gaps are filled with blank “clips”.
To insert something on a specific point in time you need to sum up the length of all the clips before that and create a blank clip in front of your “beat marker”.
Here you must pay attention to the timing conversion: i first read out the FPS of the project as a fraction and then convert between frames and time to set the right length.

So that’s practically my algorithm :wink:

Maybe there are better ways to do that. Maybe the MLT format has changed and allows for absolute timestamps. Maybe Shotcut will support markers that one can import from a file.

I can suggest using wine under linux, as it will run most VCPP static linked builds from me.

1 Like

Hello Martin, I wanted to use your solution but it does not seem to be working, I always get Read 0 producers.
I tried everything that could come up with like: sending files to the shorter filepath, using only latin letters with no spaces, having both file and _beat.txt files in the same location, reducing framerate, etc., but nothing worked

Could you, please, advise if there are any solutions to fix that?

If there are any known issues?

1 Like

I think you already did what i have had checked first. And my tool reads the framerate of your project correctly.
Even if my tool wouldn’t detect audio clips it would count all producers. But it found none.

So i assume the .mlt file format has changed as announced a while ago. (Maybe because of time remapping feature).
It’s likely that i have to adapt to that .mlt format change.

As i want to cut some videos in the next weeks i will either adapt my tool to the most recent shotcut version OR i will switch to davinci ressolve, as it offers motion tracking and speed ramping without a hassle and even color correction with my x-rite color checker.

I try to post an update about that here next week.

1 Like

Hello Martin, thanks.

Actually, I took your approach and figured out the current format of mlt file, so I created google spreadsheets file, that converts audacity file (3 columns) into xml-like format to paste into mlt file via notepad =D

Your explanation did help me a lot, I just had to use formulas to convert “absolute” seconds into “relative” (difference between ending of the last beat-shot to the start of the new one) seconds, also keeping in mind that i have to convert 1000ms into 60frames/sec, so I calculate “the exact second” from the left, to fit the exact moment, when shot should’ve started. Like, on my screenshot below, the audacity detected 28.195 would be between frames, so I use 28.183 that would be exact 00:28.11 (28sec 11 frames).
So I only have to cope and paste it inside xml (also delete if beat is more frequent that the length of a dummy shot =D), but that’s still convenient.

Again, thank you very much for your help, if I were better at programming, I might have created something similar, but since I am good at office tools and it is faster, I used excel/sheets file =D

1 Like