Ai-powered auto-editor

1

I’ve been doing some video-tutorials collaboratively during the last months (see one example here: Video-tutorials with Shotcut)

So, what we were struggling with is heavy video editing combined with the AI voiceover. I made a prototype of a tool that helps automating this process:

  • Extract audio track from the video
  • Make SRT subtitles from the track
  • Improve subtitles and brush them a little (fix mistakes, uh…ahm… and repetitions)
  • Generates an AI voiceover
  • Automatically create an MLT file where the original video track is already pre-cut, and the VO fragments are placed on the timeline.

The Final zip package with MLT and mp3 voiceover files is ready for download.

Since I’m already familiar with the MLT format, I experimented a bit with automatically baked subtitles directly into the project file and also automatically placing rawr animations (rawr files are created manually and just automatically placed on the timeline).

Here’s how the Shotcut project looks like:

image
image2241×989 351 KB

All edits were done automatically, not perfect, but significantly simplifies the future edits

Is it something interesting for the Shotcut community? I think Shotcut would benefit a lot from some of the AI-powered features that seem to be completely missing right now. However, as a non-professional video editor it’s hard to estimate if many people would benefit from it.

6 Likes
2

There is no link to the software.

4

Indeed. Right now it runs as a backend service on my VPS. I was hoping to figure out if others have similar use-case to create some kind of submission form.

6

Yes, many people do have similar use cases. I use autohotkey and python scripts to automate stuff.
It would be big relief if shotcut can have that feature.

I am up to see it :+1:t2:
Hold my vote :beer:

1 Like
7

Thanks a lot. The biggest bottleneck so far is to enable local file processing.

Right now I don’t mind uploading my videos to the VPS, but it would be quite resource-intensive for an indie project.

I found a WASM version of the ffmpeg. This would enable extracting / re-encoding of the audio stream locally. After this is done, the further steps could be made in the cloud

I need to elevate my dev skills to make this work :smile: or I need to pay someone.

1 Like
8

You can also get help from the crowd here, many people could give you an extra hand in a lot of stuff. I believe.

9

This topic was automatically closed after 89 days. New replies are no longer allowed.