MLT File Timestamps

1

I wanted to use the timestamps from an MLT file as part of an ffmpeg command, but the resulting videos are incorrect. When compared to the videos as exported by Shotcut, some of the durations and start times are off by one frame.

OS: Windows 10 64-bit (Build 19043)
Shotcut Version: 22.06.23
MLT Version: 7.8.0
The MLT file was saved with Shotcut 22.06.23.

My MLT file contains a <playlist> element that contains 8 entries from a single MKV file. My goal was to take the in and out attributes of each <entry> element and use them in an ffmpeg command to generate clips. For example:

<entry producer="producer2" in="00:13:59.351" out="00:14:05.339"/>

would map to the ffmpeg command:

ffmpeg.exe -ss 00:13:59.351 -i TEST_INPUT.mkv -t 5.988 -c:v libx264 -preset veryfast OUT01.mp4

(14:05.339 - 13:59.351 = 5.988s)

I do this for all 8 entries in my playlist, and the generated files are not what I expect. Here is a table with my results. All shifts are relative to the correct output generated by Shotcut.

           Timestamps             |    Shifts    |
    Start    |     Out    | Dur.  | Start | Dur. |
00:13:59.351  00:14:05.339  5.988 |   =0     =0  |
00:14:12.304  00:14:15.224  2.920 |   +1     =0  |
00:14:55.305  00:14:57.015  1.710 |   -1     -1  |
00:15:28.403  00:15:28.487  0.084 |   =0     =0  |
00:15:52.219  00:15:52.302  0.083 |   =0     -1  |
00:17:23.315  00:17:26.235  2.920 |   -1     -1  |
00:19:04.087  00:19:06.089  2.002 |   +1     -1  |
00:20:49.758  00:20:54.638  4.880 |   =0     =0  |

I tried this with my own installation of ffmpeg (5.1-full_build downloaded from github) and the version packaged with Shotcut (n5.0.1-5-g240d82f26e), and obtained the exact same results.

My questions are (1) is this expected, and if so (2) how do I generate timestamps that can be used with ffmpeg?

2

-t in ffmpeg is a duration, but out in MLT is the time at the start of the last frame, whose duration depends on the frame rate. duration = out - in + 1 frame. You may not like that, but that is the way it has worked for over 15 years, and not going to change it now and break many things. Also, in MLT the in and out times are interpreted using the <profile> (Video Mode in Shotcut) frame rate, but in ffmpeg the times are interpreted based on the video file’s frame rate. How to resolve this is an exercise for you. You might consider to write a custom exporter to replace Export EDL to use with ffmpeg by modifying the file in Shotcut\share\shotcut\qml\export-edl\export-edl.js.

3

duration = out - in + 1 frame

This is totally fine, and I can easily account for it. If this were my only problem, I’d see that all the durations were shifted by one frame, but that is not the case – only some are.

Also, in MLT the in and out times are interpreted using the <profile> (Video Mode in Shotcut) frame rate, but in ffmpeg the times are interpreted based on the video file’s frame rate.

The MLT framerate is 24000 / 1001 = 23.976023
The ffmpeg framerate is 33230 frames / 23:05.968 = 23.976022

The two are not different until the 6th decimal place, which is not enough to affect the video clip’s I’m trying to make. My latest timestamp is 20:54:638 = 1254.638s. Convert this to frames with 23.976023fps = 30081.22954frames. Convert this to seconds with 23.976022fps = 1254.638052s. Is there some sort of truncation/rounding I need to do obtain an integer frame count?

You might consider to write a custom exporter to replace Export EDL

I exported all my clips as an EDL file to see what the timestamps looked like. Using 1 frame = 0.0417s, I converted the frame portion of the timestamp into seconds and compared it to the MLT values. I also generated videos with ffmpeg, and the results are still incorrect but in a different way now.

           Timestamps             |    Shifts    |
    Start    |     Out    | Dur.  | Start | Dur. |
00:13:59.334  00:14:05.375  6.041 |   =0     +1  |
00:14:12.292  00:14:15.250  2.958 |   +1     =0  |
00:14:55.292  00:14:57.042  1.750 |   +1     =0  |
00:15:28.417  00:15:28.542  0.125 |   -1     =0  |
00:15:52.209  00:15:52.334  0.125 |   -1     =0  |
00:17:23.334  00:17:26.292  2.958 |   +1     =0  |
00:19:04.083  00:19:06.125  2.042 |   +1     =0  |
00:20:49.751  00:20:54.667  4.916 |   =0     =0  |

The differences between the MLT and EDL time values would seem to indicate that export-edl.js is not handling the timestamps correctly, so it’s not likely to help me with my problem.

If you’re not sure where these discrepancies are coming from, could you point me to the place(s) in the repo where timestamps are generated and used for export? I took a quick look around, but I’m unfamiliar with the organization.

4

Probably the other way around… assuming the video is 24000/1001 fps then derive the duration…

33230 frames / (24000/1001 fps) = 1385.967916667 = 23:05.967916667 duration

Always do time-based math as “number of frames since the beginning” rather than seconds. Always truncate the frame number to get an integer rather than rounding. At the very end of the frame-based math, the final frame can be converted to seconds to get the timestamp of that frame. Do not add a seconds-based timestamp to another seconds-based timestamp to calculate a new position. The precision loss in those timestamps will cause skew over time. Instead, convert both timestamps to frames, add the frames, then get the timestamp of the new frame position.

5
6

This was the key. My mistake was thinking of everything in terms of time when I should have been thinking in frames.

The correct process is:

  1. Convert the in and out timestamps to frames (mlt/src/framework/mlt_property.c at master · mltframework/mlt · GitHub, assuming lrint uses FE_TONEAREST rounding).
  2. Subtract the two frame counts and add 1 to obtain the duration.
  3. Convert the start and duration frame counts into seconds.

Many thanks for the help.

7

This topic was automatically closed after 90 days. New replies are no longer allowed.