Merging Multiple OVERLAPPING MP4 files into a Single File


Firstly a HUGE shout to the developers on this fine product…

My wife’s lab has a camera that records the lab when it is not supervised… Considering the amount of issues that usually occur overnight this is a good idea! and I need your help (HELP!) to archive a number 800+ files over 7 months into a single file per day … Each day is made up of a number of files that have a variable length and the kicker is each file can and do overlap each other and the end result is I need them merged/appended into single file per day

For example 02-11-2020 is made up of the following files…

File 1 - 02-11-2020 0236 - 1:10:05
File 2 - 02-11-2020 0345 - 0:06:35
File 3 - 02-11-2020 0408 - 0.07:32
File 4 - 02-11-2020 0411 - 0.15:32
File 5 - 02-11-2020 05:21 - 1:10:41
File 6 - 02-11-2020 05:49 - 0.25:30

Some of the files are straight appends others like File1 and File 2 are overlapping and Ideally I would like to drag 6 files onto shotcut and it merges them together… Considering the software knows the start time, duration etc this should be something I would consider Out of the Box but I cannot see how to manage the overlapping durations other than manually which is just not an option as the volume is significate.

I hope I am overlooking something fundamental and the brains in the forum can point out my oversight.

Thanks for your help!

Could you clarify how files are overlapping? Is there footage from multiple cameras?

Shotcut will not have access to the start time, so automated overlap detection can’t happen that way.

Thanks for the reply…

No It’s the same fixed camera, it was I suspect a glitch in the software or a limitation, every time an entry event got triggered it seemed to spawn a capture event.

I don’t understand your post, Why does shotcut not have access to the starting time as it is in the creation time and it is picking up the duration .

When I look at the start in the play list it is simply assuming that this is always an append rather than looking at the date stamp and detecting what is being imported has overlap, if it is a limitation in shotcut then so be it…

I meant to also say I can manually simulate putting the pieces together by manually laying down the tracks and using the overwrite clip into the current track function so the software can do what I want and this is fantastic however I cannot however do this manually it is way time consuming… I should be able to drag the days files to the play list and ask shotcut to merge them together into a single track based on date + duration not just blindly append…

I’m still not understanding how a single camera is able to create duplicate footage in two separate files recorded at two separate times, unless it has some kind of buffer that it’s emptying into the second file. But that minor detail aside…

The creation or modification timestamp of the source video files is not part of any Shotcut timeline processes. I can’t think of a single video product (commercial or FOSS) that has timeline positioning based on date/time of the source files. There is a feature called SMPTE time code that gets close to what you want. But that time code must be embedded by the camera to work (or be added later by a post-processing step), and even then the time code does not carry any date information, just time. Even if the camera wrote SMPTE time code, Shotcut currently doesn’t offer any tools to process it, although there are other programs that could.

The issue with using date/time of the source file for positioning is that date/time is not a reliable thing. If a file is corrupt and replaced by a good copy from the original camera SD card, the time stamp changes. That could mess up the timeline when it is reloaded, unless the position is locked at first load, which introduces problems of its own. Similarly, doing any in-place post-processing steps would update the date/time. Most fundamentally of all, using proxies during editing could severely mess up positioning because the date/time of the proxies would be used instead of the originals. Long story short, I would not expect a file’s date/time stamp to be out-of-the-box processing for any editing product. It’s too unreliable for the average user, even if you individually promised to be super careful while using it.

Realistically, what’s the longest overlap? If it’s only a few seconds, is it a deal-breaker to include the overlap and reduce editing to simple appending?

If eliminating the overlaps is critical, then Shotcut may not be the fastest tool. It could be possible to write a custom script in Python or PowerShell that uses the date/time stamp and grabs the duration from ffprobe to build a string of ffmpeg commands that builds a final file for you. Or, you could find a utility that generates SMPTE time code starting at the file’s time stamp and embeds it in the video files so they can be processed in another tool like Premiere or Resolve.

Aside from those options, I can’t think of anything else that would be straight-forward and fast.

It’s not the camera, it is the capture process that is doing this however it does not matter what did it as I got the issue and it needs to be solved and FYI unfortunately the overlaps are variable some a minute or two others many minutes long sometimes the over laps start and extend longer than the original script or in some cases the entire clip can be deleted as duplicated in another clip.

Thanks for the tip, I also stumbled on a posting around ffmpeg and I am constructing a excel file so I could write a bash script to do this for me although my bash skills are crude for this level but I can at least trim clips so I can do an append process…

Personally I don’t agree with your POV on timedate stamps. We live in the age of GPS timestamps yes as a photographer I know the issues around date/time stamps in EXIF information unfortunately too well but working on the assumption the timestamps are incorrect I would argue is faulty based on our prior experiences from other decades when they got manually set. I have workflows that are time based, light is critical for certain types of shots and thus light is very much time based so I always been bemused when importing clips to be presented with a timescale starting from 0:00:00 rather than a timescale based on the starting date/time stamp of the clip but this is my workflow requirements clearly not shared by the majority of others.

The frustrating part for me, is I know shotcut is aware of the timestamps as when you import the clips into the playlist it is clearly taking them into account to sequence the clips and shotcut start position is then based on the duration of the clip based on the sequence of clips in the playlist.

In my use case the assumption of an append in the start calculation is the issue, if this behaviour could have been influenced to either toggle append/merge on subsequent clips then my use case would have been quickly solved.

(Yes I know the clips will not merge seamlessly as it would be almost impossible to match the frame (30 frames a sec so good luck in calculating that) but I accept a slight 1 second overlap rather than miss footage that might be critical (and I am not going to watch hundreds of hours to determine this))…

Thanks again for your support, I like the product as it’s clearly lovingly been developed and shows it’s maturity and the product is backed up by people like yourself with your deep subject matter expertise that creates a sense of community here in this forum…

This topic was automatically closed after 90 days. New replies are no longer allowed.