Different Frames when using cuts inside Shotcut and in the Java FFmpegFrameGrabber class

1

I got an video, which is 14059 frames long. If I use it inside my java application and put a cut into frame 369 (at least the FFmpegFrameGrabber method getFrameNumber() says so), and then I put this cut into a .file and open this one in Shotcut, the frame is 3 frames off from where the cut should be. I found out, when I this:

grabber.setFrameNumber(0);
Frame frame;
while ((frame = grabber.grabImage()) != null){
    if (frame.image == null) continue;
    BufferedImage wholeFrame = converter.convert(frame);
    if (wholeFrame == null) continue;
    System.out.print(grabber.getFrameNumber() + ", ");
}

it prints: 2, 2, 3, 3, 4, 4, 5, 6, 7, 8, 9, 10, 11, …, 2866, 2867, 2869, 2870, …, 6880, 6881, 6883, 6884, …, 10897, 10899, …
So you see, there are frames missing and some frames are duplicated. If I put the exact same video into Shotcut, and export it without any changes and do the same, it prints:

0, 1, 2, 3, 4, 5, …

So what you would normally expect. The problem is, that it’s not just different numbers, but the first frame in the original video is the same as in the expected one. Even if the frameNumber in the first frame is different, it shows the same content. But a the third frame already is one frame off from the third frame in the exported video. This changes again later. What is the problem, is there a way to check in the java code, how many frames the shotcut video would be off from the original? The same problem is already if I just set a cut inside Shotcut in the original video without exporting it: the .mlt file says the cut is on frame 366 but my code says 369 for the same content of the frame. I hope you get what I mean.
I could fix these offsets by always exporting the video without cuts in Shotcut, then applying my cuts through my code and create a new .mlt with these cuts on the exported one and then the cuts are on the correct position. But I don’t always want to wait minutes or hours to export a video before I can apply cuts through my automated code to see where my code set the cuts.

2

Is the source video variable frame rate? I haven’t tested to know if that would cause the dupe/drop frames, but it would be nice to eliminate VFR as a possible problem.

3

AFAIK Shotcut always works in constant framerate therefore it deals with variable frame rate by using the nearest frame (time-wise) from the original source, so it either duplicates or skips frames as needed. If you instead process a video frame by frame you ignore the timings therefore the results will definitely be off by a few frames for the portions where the real frame rate is being variable enough to skip the correct time.

As a guess I’d say you’ll need to compute the Shotcut expected frame time (using some sort of base frame rate that you are also using in Shotcut’s project Video Mode) instead of just assuming frame X is at (real_fps) * X for every frame.

4

To make things even more fun, the “encoded sequence of images” and the “display order of images” are not necessarily the same thing. The encoder can reorder frames, then instruct the player to display them in a different order if it thinks that will help keep bitrate more controlled. In particular, the encoder may put later B-frames right before or after an I-frame so that the small B-frames can offset the large I-frames in terms of average bits per second.

5

Windows says, it has 29.97 fps. But another tool says it has VFR (minimum of 26.834 and maximum of 60.000 FPS). Is there an example where I can see how I can compute the Shotcut expected frame time? If now I probably really have to export and then use my code on the result, to export it again.

6

This is probably the best way. Shotcut refers to this as “Convert to edit-friendly”, which means to make an optimized version that removes VFR among other things. Precise cuts are almost impossible to guarantee when using multiple unrelated tools on VFR footage, as there are different methods to handle missing timestamps, not to mention different ways of having floating point errors between tools.

This step could be automated by a script that calls ffmpeg.

7

This topic was automatically closed after 90 days. New replies are no longer allowed.