What is your operating system?
MacOS Sonoma (14.1.1)
What is your Shotcut version (see Help > About Shotcut)?
Shotcut version 23.09.29
Can you repeat the problem? If so, what are the steps?
I’m trying to produce a video of a 45 minute conference talk. The speaker should “fit” into a smaller part of the screen and I use motion tracking to create the locked on motion tracking effect for this.
My problem is that the motion tracker (regardless of the algorithm) creates apparently so many keyframes, that Shotcut doesn’t handle it well:
The motion tracker analysis takes about 2-3 hours on my Macbook Pro M1 - the only good part about this is that there is a progress indication in the jobs queue, CPU usage is about 200% and Shotcut doesn’t manage to use the full power of my system (8 cores).
Importing the keyframes from the motion tracker to the “Size, Position, Rotation” effect takes also about 1h, but there is no progress information, the mouse curser is the colour ball and CPU usage is 100% - so Shotcut doesn’t use more than 1 core for this task.
After the keyframes are imported, Shotcut is practically unusable:
Clicking anywhere or moving anything around causes minute-long appearances of the Mac colour ball. This is especially true for seeking in the timeline, changing timeline zoom level, clicking anywhere in the keyframes view, scrolling the filters view and many other places of the Shotcut UI
I needed to add another Size, Position, Rotation effect to move the motion-locked video to the actual position on screen. I wasn’t able to do so because I can’t move the rectangle tool to the right place. Each time I start moving I get only the colour ball and the rectangle doesn’t move.
The MLT file increased in size from 35 KB to 1.2 MB, maybe this gives an indication of the problem dimension.
In the end I’m not able to create my video as desired.
For reference, I managed to produce my video in Kdenlive (with crashes and other problems), you can look at the result at https://www.youtube.com/watch?v=K7WrOjY3CYA to get an impression of what I’m trying to do.
I’m reporting this bug and asking you to improve Shotcut for handling large amounts of keyframes effectively:
queuing, progress indicator and multi-core support for operations that take longer, e.g. importing key frames
support large amount of keyframes
maybe add support for fine-tuning the motion tracking. In Kdenlive I got this to work only after reducing the motion tracking granularity to 30 keyframes, so that the total resulting keyframes was about 3000.
Maybe a workaround could be to automate the splitting of a longer video clip (45 mins in my case) into shorter segments for motion tracking & Size, Position, Rotation adjustment - but it would have to support seamless gaps between the segments which is also the reason I didn’t try to employ this workaround manually.
Otherwise I’m happy to report that using Shotcut was significantly simpler compared to Kdenlive, mainly because I can do all filter adjustments directly in the main video result view.
This falls under the extreme usage category. I have no near or mid-term plans to improve this.
That is highly unlikely, but you can do this manually. And I would even export each segment to an intermediate format to include as a simple clip in the larger composition.
I also was not able to use the motion tracker on short clips and feed the zoom filter. Shotcut crashed every time it was (almost?) finished with the motion tracking.
Is the presenter moving so fast, all the time that you can’t set up the keyframes yourself manually? Motion tracking adds keyframes at a very high frequency, which is probably too precise for your use case
Yes, the presenter is walking around the stage. Walking, not running. Which is why I managed to solve this in Kdenlive via tuning the keyframe spacing so that the total amount of motion tracking keyframes was about 3000. Loading that into the Transform filter was also a problem (Kdenlive “stuck” for like 20 mins), but after that it worked.
Shotcut doesn’t provide fine-grained controls over the motion tracking and also doesn’t handle the large amount of resulting keyframes.
I think that the status quo is a real problem because it leaves the user without any guidance
That would be lovely! I think in terms of guidance it would really help if you could figure out a maximum amount of keyframes that Shotcut can “safely” handle and make it simple to adjust the keyframe frequency in a way to not exceed that maxium number.
For shorter clips that means very precise motion tracking and for long clips it means that motion tracking becomes less precise, but Shotcut keeps working and it provides a clear guidance to the user.
Then, maybe in the future, you can increase the amount of keyframes that Shotcut can handle to improve motion tracking also for longer clips.
That is good news
By the way, do you also intend to allow higher keyframes frequency?
The actual 1 keyframe every 5 frames is good enough for most cases, but in may occasions I would have liked to get 1 keyframe every 2 or 3 frames… and sometimes even 1 keyframe per frame.
A keyframe interval option would be great. From looking at the XML for a motion track I recently did, it appears to default to 6 keyframes per second. In my case, I’ve found good enough smoothness for manual keyframes with the frequency of 1 per second or in some cases 1 per 5 seconds. If that would make the Motion Tracker run anywhere from 6 to 30 times faster, this would be a GAME CHANGER! Allowing the user to set the frequency would be a huge improvement.
I don’t consider the user’s case as “extreme” or unusual, but that’s just me. I’m in the same use case as him. I have 20 to 45 minute videos I want to motion track because I do hiking videos, and I want to be able to keep the walking subject in the center of screen. Currently the Motion Tracker is way too slow for longer clips. I’m glad to have read this too, because I was considering Motion Tracking longer clips. If that number of keyframes kills the app, then I won’t bother. It would have taken 16+ hours to complete the track, and to know it might crash near the end… uh nope. I won’t be doing that.
Literally anyone doing outdoor content, especially using an action cam, will potentially want this ability to motion track moving subjects over longer periods of time. So it’s not necessarily extreme, except that action cams content can be called “extreme” in another pun sense.
I realize there is crossover between Stabilize and Motion Tracker use cases. My case is a little unusual in that I have 360 footage that I need to use a Motion Tracker to “stabilize” the position of myself walking in the videos. There is no built-in way to do this in Shotcut, but I was trying to use the Motion Tracker to get data to feed into the 360 Transform filter. It would have worked too, except I don’t have the patience to wait for the Motion Tracker’s many hours of processing time. But, if the frequency of keyframes could be changed, and thus the processing time fractioned, then it could be a viable path to get what I need.
Just doing some basic math here. Estimated time for 46 minutes of 5.7K footage to analyze Motion Track = about 16.8 hours, give or take. If I could set the keyframe interval to 1 per 5 seconds, instead of 1/6 second, that would take the processing time down from 16+ hours to less than 1 hour! It would also make Shotcut more stable, since fewer keyframes means it gets less bogged down.
