V21.04 ALPHA/UNSTABLE - Time Remap

Can you pick up some Almond milk?

1 Like

Yuck! But sure.

2 Likes

Ooooo, really good point. If the line between keyframes changed color for the 1x sections, it would be super easy to tell at a glance which segments were 1x and which were not. I understand this may break the generic rendering of the keyframe system, but weā€™re dreaming, right? :grin:

1 Like

So is the slope in the first keyframe and last keyframe supposed to be identical? (45 degrees)

Supposed to be, yes. When I stretched the original image, the keyframe diamonds stretched too. I copy-and-pasted normal diamonds back in, though not in the perfect locations. So the slopes are a little wonky. Sorry, I just mocked-up something fast for illustrative purposes, but yes, I can see that would be very confusing if it were official documentation.

1 Like

Thatā€™s what threw me off

1 Like

That screenshot is inspiring - and I agree: a good visualization would help with the intuition. The mockup image you provided builds that visualization in the keyframes panel. Currently, we have one generic keyframe implementation that all filters must fit within. I think that to generalize this, we could add a future feature where a filter can provide a ā€œcustomā€ keyframe view implementation that can override the generic view we have today.

Until we have that advanced capability in the keyframe panel, does anyone have any ideas for simple visualizations that we could put in the filter panel instead?

This is a promising idea. And I think it could be generalized to other filters. For example, the gain filter could have a watermark line at 0dB to indicate were a keyframe would be placed to make no change to the audio.

The new buttons I added should help with this since the user can not set a keyframe to achieve 1x speed.

One comment I would add about the visualizations brainstorming: it might be nice to label the space between two keyframes with a speed. But donā€™t forget about smooth keyframes. If the keyframe type is ā€œlinearā€, then the speed will be constant between two keyframes. But if the keyframe type is ā€œsmoothā€, then the speed is changing with each frame.

1 Like

Nice!

Hello @brian
Iā€™m wondering if making gradual speed changes is on the road map.
Using the Smooth keyframe type doesnā€™t quite work because the line curves bellow 1.00000x and makes the video go to reverse for a brief moment.

1 Like

It can help to not make every keyframe type smooth to gain more control over the curvature. In your smooth example, to prevent the portion between keyframes 2 and 3 from going into reverse, change keyframe 2 to linear. Then, it will still curve into and out of that portion. See also

Iā€™ve tried every configurations.
It seems to work when I set only the third keyframe (from slow speed to normal speed) to smooth.
If I set only the second keyframe (from normal speed to slow speed) to smooth, it makes the video go briefly to reverse.

Iā€™m not reporting a bug. Iā€™m just wondering if there will be a way to make gradual speed transition in the Time Remap filter.

Re-read my message. I updated it while you were typing. The type of keyframe before a line segment controls its interpolation.

I think you should be able to do it now. I suggest to try adding a couple of intermediate smooth keyframes between the two in order to coerce the line the way you want and avoid the line going past zero.

Sorry to both of you if I often look like the dumb student in the classroom. My poor excuse is the trouble I have sometimes to clearly translate in English what I try to say.

Using the actual keyframe type settings to make smooth speed transition might work, but finding the right way to set the keyframes will be different each time. Depending on if you speed up, slow down or freeze the video, it will involve each time some amount of tries and guesses.
My video above was just an example to illustrate what I meant with my original question. Iā€™ve tried a lot of ways to make it work, and I will try even more based on your suggestions.

So for now I guess the answer to my question is: The only way to make smooth speed changes is using the Keyframe type settings, and nothing else is planed as an alternative way.
Thank you :slight_smile:

Correct.

But if you find a scenario that can not work well with this, then we should explore it further to see if we can improve it.

Based on UNSTABLE-21.02.25
6) The speed is not starting at 1.00000 now. It starts at a speed past it. With this clip itā€™s starting at 1.00028x:

images2.imgbox.com/4c/a5/sdtZJ8aB_o.png

7) Iā€™ve experienced the same thing that @MusicalBox is reporting with the Smooth keyframe. The behavior of it has to be looked at because itā€™s too erratic to the point of being useless as itā€™s so unreliable. This doesnā€™t happen on other keyframes with curves. Iā€™m expecting a smoother transition between two speed points but sometimes it just decides to reverse footage and go forward again and there is nothing on the curves to show that itā€™s doing that. To reproduce this right after adding the Time Remap filter add three keyframes that are about 4 seconds apart from each other. Do not alter the time or speed on any of them. Just add those three keyframes. Then change the second of those three keyframes to smooth. When you play it back and it gets closer to that second keyframe you will see it reverse for no reason even though no change to speed or time was applied. Here is a demo.

Notice in the demo you can see the aforementioned issue of the speed not defaulting to 1.00000 as the clip has the speed at 1.00018x as its default.

8) Simply adding the Time Remap filter adds crackling sounds to the audio. This can be heard in the demo I just linked to above.

9) Iā€™m not sure what the filter trim buttons are doing now but they behave strange. If I put the playhead right in the middle of the default start and end keyframes and press the left filter trim button then I press the reset button in the time parameter, even if it looks like the start keyframe is at its default position itā€™s actually frozen. So it ends up that the first half is frozen.

10) Cutting off the start and end of a clip with I or O makes the keyframe timeline end up looking like this:

images2.imgbox.com/8e/81/zyX2QJpk_o.png

11) If you take a clip with the Time Remap applied then strech the end of the clip out to make it longer the result is that the last keyframe is held there as a freeze like this:

images2.imgbox.com/32/e1/x1RHamtN_o.png

Likewise if you strech out the start of the clip then the first keyframe also is held as a freeze like this:

images2.imgbox.com/30/17/DQwEl2iJ_o.png

I think what would be better is that after the strech all the keyframes get their heights dynamically updated to match the time since the frames that have been marked on the keyframes would not be in the same height as per the time depending on the strech. If someone is using this filter decides after setting some keyframes to then make the clip longer because they want to grab a frame from earlier or later to apply to their keyframes with the current way its set they would have to reset the whole keyframe timeline and redo everything.

12) It seems that adding Time Remap alters the colors/tint on a clip.

Frame without Time Remap:

images2.imgbox.com/0a/3e/SXPVHvrZ_o.png

Frame with Time Remap:

images2.imgbox.com/0a/59/MUZVZhII_o.png

The blues in the sky are darker in the original frame without Time Remap.

13) Since you are going to go ahead and debut this filter even with the B frame issue I think that there should be an additional option added to the Convert To Edit Friendly feature. Right now Convert to Edit Friendly converts the entire source file. Most use cases for Time Remap are not going to be using the entire source file especially if a video with a long duration being used. Itā€™s usually just one section or a couple of sections that are used from a longer video. So how about a new added option that gives the user the ability to just use Convert to Edit Friendly on only a selected section instead of converting the whole source file similar to how the Reverse feature only reverses a selected section? So on Convert to Edit Friendly the user can get to pick between two options: Convert the whole source file or convert only the selected section.

With that in mind in order to avoid the big hoop of having to do Source > trim > Export > intermediate or lossless preset > Export File > double-click completed job to do only a selected section, how about when you pick the the Time Remap filter and the dialog comes up instead of it just telling the user that the clip is not compatible and that they have to go ahead and do every step themselves to convert the section they want, that dialog would ask the user if they want to convert the clip to use the filter. When they say yes then they are taking to the Convert to Edit Friendly option to convert just that selected section.

If this filter is going to go ahead anyway then I think that this is the best way to avoid having to go through all the previous suggested steps to get a compatible clip going. However, the one limitation to this is that once this clip is converted what is in the timeline now is limited to just the start and end of the newly converted clip. Since itā€™s not coming from the source file this means that if the user during their use of Time Remap wants to stretch the clip more on either side or both sides to have more footage to work with they wonā€™t be able to do it. Iā€™m wondering if some kind of an additional option can be created to give the user the chance of going back to the original source file of that selected section to pull more footage from it and convert that to a new file to replace the last Convert to Edit Friendly selected section.

14) When the Blend option is picked in Image Mode it applies it regardless if the footage is going faster, at regular speed or slow. Can there be an option added to only apply Blend if the footage is going faster or slower than 1.00000x?

By the way, could the Blend option be added to Properties near the Speed parameter?

1 Like

@Brian we will have to disable smooth keyframes on this filter. It is impossible to change the interpolation algorithm to avoid these unexpected scenarios.

I was designing in a different venue some time back, I I ran into the same problem using cubic curves.
In that case, I found the by swapping which thing I called ā€œxā€ and which I called ā€œyā€, flipping the cubic equation on it side, as it were, I still got a smooth cubic curve between the two central points of my four points, albeit a somewhat different curve, but dy/dx then never went from positive to negative (or vice versa) within that range, neither did ā€œyā€ ever exceed the high/low range of the two center points.

Sorry but that does not help much. If you want to add a new interpolation algorithm and figure out how to serialize and deserialize in a unique but compatible manner, then you need to make a code contribution to MLT. It also should be able to transition between yours, linear, and discreet/hold like the current one does.

1 Like

I suspected it might not fit the application at hand, but I thought possibly it might; which is why I framed it as ā€œthis worked for meā€, trying as best I could to avoid implying ā€œyou shouldā€¦ā€.