Shotcut unusable with large amount of Keyframes for 45-min motion tracking

Bug
,
1

What is your operating system?
MacOS Sonoma (14.1.1)

What is your Shotcut version (see Help > About Shotcut)?
Shotcut version 23.09.29

Can you repeat the problem? If so, what are the steps?

I’m trying to produce a video of a 45 minute conference talk. The speaker should “fit” into a smaller part of the screen and I use motion tracking to create the locked on motion tracking effect for this.

My problem is that the motion tracker (regardless of the algorithm) creates apparently so many keyframes, that Shotcut doesn’t handle it well:

  • The motion tracker analysis takes about 2-3 hours on my Macbook Pro M1 - the only good part about this is that there is a progress indication in the jobs queue, CPU usage is about 200% and Shotcut doesn’t manage to use the full power of my system (8 cores).
  • Importing the keyframes from the motion tracker to the “Size, Position, Rotation” effect takes also about 1h, but there is no progress information, the mouse curser is the colour ball and CPU usage is 100% - so Shotcut doesn’t use more than 1 core for this task.

After the keyframes are imported, Shotcut is practically unusable:

  • Clicking anywhere or moving anything around causes minute-long appearances of the Mac colour ball. This is especially true for seeking in the timeline, changing timeline zoom level, clicking anywhere in the keyframes view, scrolling the filters view and many other places of the Shotcut UI
  • I needed to add another Size, Position, Rotation effect to move the motion-locked video to the actual position on screen. I wasn’t able to do so because I can’t move the rectangle tool to the right place. Each time I start moving I get only the colour ball and the rectangle doesn’t move.

The MLT file increased in size from 35 KB to 1.2 MB, maybe this gives an indication of the problem dimension.

In the end I’m not able to create my video as desired.

For reference, I managed to produce my video in Kdenlive (with crashes and other problems), you can look at the result at https://www.youtube.com/watch?v=K7WrOjY3CYA to get an impression of what I’m trying to do.

I’m reporting this bug and asking you to improve Shotcut for handling large amounts of keyframes effectively:

  • queuing, progress indicator and multi-core support for operations that take longer, e.g. importing key frames
  • support large amount of keyframes
  • maybe add support for fine-tuning the motion tracking. In Kdenlive I got this to work only after reducing the motion tracking granularity to 30 keyframes, so that the total resulting keyframes was about 3000.

Maybe a workaround could be to automate the splitting of a longer video clip (45 mins in my case) into shorter segments for motion tracking & Size, Position, Rotation adjustment - but it would have to support seamless gaps between the segments which is also the reason I didn’t try to employ this workaround manually.

Otherwise I’m happy to report that using Shotcut was significantly simpler compared to Kdenlive, mainly because I can do all filter adjustments directly in the main video result view.

2

the video illustrates how Shotcut is unusable after it managed to load the keyframes.

3

This falls under the extreme usage category. I have no near or mid-term plans to improve this.

That is highly unlikely, but you can do this manually. And I would even export each segment to an intermediate format to include as a simple clip in the larger composition.

4

I also was not able to use the motion tracker on short clips and feed the zoom filter. Shotcut crashed every time it was (almost?) finished with the motion tracking.

5

Unable to reproduce.

6

This is really really sad to hear. Can you suggest an alternative solution to use for motion tracking a video longer than just a few minutes?

Is motion locking the speaker of a lecture recording really “extreme usage”? I’m asking because I’m not doing this professionally but only seldomly.

1 Like
7

Is the presenter moving so fast, all the time that you can’t set up the keyframes yourself manually? Motion tracking adds keyframes at a very high frequency, which is probably too precise for your use case

2 Likes
8

I will try to add a keyframe frequency in the version after next, and maybe a progress dialog.

4 Likes
9

Yes, the presenter is walking around the stage. Walking, not running. Which is why I managed to solve this in Kdenlive via tuning the keyframe spacing so that the total amount of motion tracking keyframes was about 3000. Loading that into the Transform filter was also a problem (Kdenlive “stuck” for like 20 mins), but after that it worked.

Shotcut doesn’t provide fine-grained controls over the motion tracking and also doesn’t handle the large amount of resulting keyframes.

I think that the status quo is a real problem because it leaves the user without any guidance

10

That would be lovely! I think in terms of guidance it would really help if you could figure out a maximum amount of keyframes that Shotcut can “safely” handle and make it simple to adjust the keyframe frequency in a way to not exceed that maxium number.

For shorter clips that means very precise motion tracking and for long clips it means that motion tracking becomes less precise, but Shotcut keeps working and it provides a clear guidance to the user.

Then, maybe in the future, you can increase the amount of keyframes that Shotcut can handle to improve motion tracking also for longer clips.

Much appreciated!

11

That is good news :+1:
By the way, do you also intend to allow higher keyframes frequency?
The actual 1 keyframe every 5 frames is good enough for most cases, but in may occasions I would have liked to get 1 keyframe every 2 or 3 frames… and sometimes even 1 keyframe per frame.

3 Likes