Creation time multi-file/batch update (in playlist)

[daniel47]

Due to timezones (mostly), when adding clips shot with different cameras they are not chronological in the playlist so I have to manually get through them either one by one or do some sort of “insertion sort” if one source has few clips.

I propose as there’s already a “Set creation time” entry on right clicking a playlist item, to add the feature to the available options to choose something relative when having multiple entries selected. What I mean is in this menu:

To add (or replace it altogether when multiple items are selected) a “Relative offset” drop down option and on the bottom area to be able to input “hh:mm:ss” (which will obviously be applied for each item, relative to it’s own original creation date).

This would be massively helpful as I assume other users are also dealing with bad datetimes (both due to timezones and bad input time when setting cameras).

[brian]

Do you have files that have a timezone specified in it? If you do, I could use it as a test case to make sure timezone offset is accounted for. Every test file I have ever seen has the time saved in GMT.

Assuming that you do not have files with timezones other than GMT, your best solution is to make sure your cameras all have the correct time set. This should happen automatically for cell phone cameras. But maybe you have to set it manually for other kinds of cameras.

Another question: when you use the “Set creation time” feature, in the drop down, is there a time option that is correct? Shotcut has multiple time sources to choose from and has to make its best guess if they do not match. Those options are provided in the drop-down selection. Maybe Shotcut is not guessing the best one.

Here are the options in one of my test files:

You can see that the “System - Modified” time and the “Metadata - Creation Time” match.

I am interested to know more about this use case. Can you tell us more about the cameras you are using and the process for them to have the time set?

[daniel47]

I want to mention that I am not saying this is a bug in Shotcut, the problem clearly lies with the cameras themselves storing the time in different ways but I think Shotcut could be a great time saver with a small enhancement.
About timezones not in GMT, wel… this is complicated, I’ll detail this below but I live in GMT+2 (or +3 in summer), and there’s also travelling which complicates non internet/automatic time cameras.
The values in the dropdown… well again, it’s complicated, it has the correct time for some of the cameras, but not for the others, and I don’t think this works if multiple clips are selected (in playlist!).

---

It’s also important to allow multi select-aware options (ie: just add 2 hours to the currently detected creation time of each selected clip), as having to go one by one for 30-50 video files would defeat the purpose of efficiency. I think this part is enough to ignore all the details below as it doesn’t matter what datetime is detected automatically, it will be clearly consistent for one camera type and the user can just at the time and say: this is 2hours ahead of this other file I consider correct, I just want to substract 2 hours from what I see and don’t care about timezones or proper fields.

---

I’ll describe ~4 types of files (cameras) I use very often and the result vs expected:

1. Android phone //mostly good
   The phone stores the time properly (in EXIF in UTC, and it is read and displayed correctly), and not only that it stores local datetime in the filename so it is easy to double check this.

• small caveat: phone stores creation date at the end of the recording (all other cameras use the time at the start of the recording) - this is a very small issue and I don’t want to overcomplicate things with it
• another small caveat: when travelling to another timezone, untill I get a local SIM card (or roaming connection) the old timezone is still used so until the phone can sync the time correctly all videos will be in the wrong time. After sync, depending on timezone direction it will either have a gap (no real issue here) or it will overlap older time making “chronologic” sorting basically impossible. - This is clearly a camera issue that can’t be fixed automatically by shotcut but can be manually by adding/substractic the timezone difference for those files.
  
  

2. GoPro/basic cameras with no timezone in settings //the biggest issue for chronology
   This one is pretty straightforward, it will only be correct if you live in timezone 0 without DST and don’t travel to another timezone, any other person will see wrong date*.

• weirdly, as you can see in my screenshot, one of the fields (in Set creation time) is the correct one (I guess it assumes local time instead of UTC as a hint or unintentional) but due to the wrong reasons, media info clearly states 18:14:05 UTC
• *technically you can intentionally set the camera time to be the one in UTC but who does this, it’s 12:50 right now for me (local time) so I’d need to set the gopro time to 09:50 and do mental math every time I look at the clips info
• one extra thing I’d like to mention is the gopro doesn’t have a way to set seconds and also over time it loses track of time slightly, this is why my suggestion is to be able to set “hh:mm:ss” instead of just hours (and also apparently there are timezones with fractions of an hour).
  
  

3. .MTS files from a sony camera with timezones //good
   I want to mention this one specifically because Shotcut can’t read .MTS metadata (I can’t even click on the metadata tab in properties) so even though the camera knows about timezones (and mediainfo can read it correctly with +3:00 at the end - so I guess this answers your question about timezones not always stored in UTC), it will use the System modified date - which is I guess the correct result but in an unexpected way

• Note I don’t really care about this metadata, I know sony does it’s own thing and it’s an .MTS file.
  
  

4. DJI drone //good
   This does pretty much everything right, I think it automatically gets the time from GPS directly.

• only reason I wanted to mention this is because it does the [sort of] opposite to what the android phone does regarding start/stop time: the creation time metadata has the time when I start the recording but the file Modified time has the time the recording is stopped. It is funny how arbitrarily this is chosen.
  
  

This has been a long post, I think it clarifies most of the questions.
So yes, basically they could all be fixed externally/before the recording but I think Shotcut could really fix this after the fact in a quick manner if allowed to multi/batch relative update of the times.

[brian]

Thanks for the examples. This is helpful for me to better understand the use case.

Before I go on, I want to clarify one point that you probably already know. UTC is the only time that is the “real” time. UTC = GMT = Zulu (Z) time. They are all the same. Time zone information is only a modification to the display of the time.

Here are some comments that might help both of our understanding.

1. Phone

@daniel47

when travelling to another timezone, untill I get a local SIM card (or roaming connection) the old timezone is still used so until the phone can sync the time correctly all videos will be in the wrong time.

This statement does not match my expectation. The phone always stores the time information in UTC. And when you change time zones on the phone, UTC does not change as stored in the metadata. However, changing time zones might change the way the phone chooses to display the date in the file name (as you mentioned). Also, for the file modification time, the display in the windows properties panel will depend on the time zone of the computer displaying the properties. This is illustrated in this picture:

When determining the creation_date, Shotcut prefers the metadata over the file modification time.

2. GoPro

@daniel47

GoPro/basic cameras with no timezone in settings

If this is the case, my advice is to set the time on all these cameras to UTC time (no matter what time zone you are in)

@daniel47

it will only be correct if you live in timezone 0

If you set the time to UTC, it will always be correct no matter what time zone you are recording in. However, the camera operator may need to do some mental math to convert to local time if they need to use the camera’s time for some reason.

4. DJI

@daniel47

This does pretty much everything right, I think it automatically gets the time from GPS directly.

If it gets time from GPS, then the time is UTC - probably why it always works.

Your suggestion is valid and captured well in this post. However, I think it is unlikely that anyone will implement it. And if someone does implement it, I do not know if it would be accepted into the code base.

I am, however, interested in two things:

1. Creating a guide for our users to help them know how to set the time on their devices for the best results (your writeup in this post is a good start)
2. Improving the time detection in Shotcut so we get the best results.

As an example for #2, we added some special logic for iphones because some Apple devices re-encode the files after the recording is finished and then the metadata changes. So we prefer the “apple.quicktime.creationdate” metadata over the “creation_time” metadata.

Maybe other devices have similar tricks that we do not know about.

In general, the calculation for the date goes as follows:

1. Use the user set creation date (if present)
2. Use apple.quicktime.creationdate metadata (if present)
3. Use creation_time metadata (if present)
4. Use the file modification time (as a last resort)

If you find some devices for which the time is set correctly, but Shotcut does not detect the time you expect, you can share the file with me and I can try to make it work better. Also, you can inspect the metadata tab in the properties panel to try to interpret where the time determination is going wrong.

[daniel47]

@brian

If this is the case, my advice is to set the time on all these cameras to UTC time

I was afraid you’d say this.
Having an in-software fix would be useful for more users as I don’t think a lot of people set UTC time just to make sure timezones are not interfering with their workflow. Not to mention very few people would think about this beforehand so setting UTC is only a future fix.
It would also help read-only file locations. In my case I’ll probably just make a python script to fix this on a per file/folder basis.

@brian

However, changing time zones might change the way the phone chooses to display the date in the file name

Oh, this is my bad, I now remember this happened to .jpegs for which I can see Shotcut reads the file modified date as creation (and ignores EXIF info) and/or I probably used the default sort by filename.

[DDer_Joel]

hi there, I jut came across this and have the same issue.
TLDR; tell shotcut to interpret the times from UTC: Open shotcut with TZ=UTC shotcut

When I normally open shotcut, and read my File from my GoPro; shotcut will display the date as xxx 02:35:04. But I created the file at 2:35PM in Timezone Auckland (which has a +12hrs) When I now start shotcut with the TZ environment variable set to UTC with the command: TZ=UTC shotcut (probably need adjustment for non-Linux) then shotcut will display the 14:35:04 in the playlist date column.

(I was just annoyed about the gopro files to be wrong, but after some research I realised that they are not. They put the correct date of 14:35:04 into the exif block (e.g. Media Create Date, Track Create Date, Create Date) but Shotcut does not parse them as I’d expect)

Cheers

[brian]

@DDer_Joel

They put the correct date of 14:35:04 into the exif block

Shotcut does not use EXIF. It uses other metadata fields encoded in the file.

If the “correct date” is 14:35:04 UTC, and you set your timezone to UTC, then Shotcut is displaying the correct date.

Can you share a screenshot of the Metadata tab in the properties panel for your clip? That will show us the exact metadata properties that Shotcut has available.

Here is an example from one of my clips:

[DDer_Joel]

Sure thing. Shotcut is opened with TZ=UTC parameter

TZ=UTC shotcut

[DDer_Joel]

And here Shotcut opened without the parameter. (I’m not allowed to put them into one post)
TZ= shotcut

the file on the file system is

$ ll GX013673.MP4 
-rwxr-xr-x 1 joel joel 244858605 Apr 14 02:40 GX013673.MP4*

and my local settings are

$ timedatectl 
               Local time: Tue 2024-05-07 11:49:21 NZST
           Universal time: Mon 2024-05-06 23:49:21 UTC
                 RTC time: Mon 2024-05-06 23:49:22
                Time zone: Pacific/Auckland (NZST, +1200)
System clock synchronized: no
              NTP service: active
          RTC in local TZ: no

[DDer_Joel]

Also, I noted that then I load clips from my iPhone, everything is scrambled up.

I’ve got two clips IMG_3530.MP4 and IMG_3531.MP4

$ exiftool  IMG_3530.MP4 |grep 2024 
File Modification Date/Time     : 2024:05:06 21:31:46+12:00
File Access Date/Time           : 2024:05:07 10:19:21+12:00
File Inode Change Date/Time     : 2024:05:07 09:34:17+12:00
Create Date                     : 2024:04:14 12:52:29
Modify Date                     : 2024:04:14 12:52:29
Track Create Date               : 2024:04:14 12:52:29
Track Modify Date               : 2024:04:14 12:52:29
Media Create Date               : 2024:04:14 12:52:29
Media Modify Date               : 2024:04:14 12:52:29
Date/Time Original              : 2024:04:14 09:47:33+12:00

$ exiftool IMG_3531.MP4 |grep 2024 
File Modification Date/Time     : 2024:05:06 21:31:44+12:00
File Access Date/Time           : 2024:05:07 10:19:18+12:00
File Inode Change Date/Time     : 2024:05:07 09:34:17+12:00
Create Date                     : 2024:04:14 10:10:52
Modify Date                     : 2024:04:14 10:10:53
Track Create Date               : 2024:04:14 10:10:52
Track Modify Date               : 2024:04:14 10:10:53
Media Create Date               : 2024:04:14 10:10:52
Media Modify Date               : 2024:04:14 10:10:53
Date/Time Original              : 2024:04:14 11:08:19+12:00

I’ve downloaded those clips via iCloud-zip and extracted them just earlier today (hence the ‘File Access Date/Time’ from today). When I see those clips on my phone it displays ‘Date/Time Original’, which matches with when I actually took them. 9:47AM and 11:08 AM in Pacific/Auckland (+12h)

When I load those clips into ShotCut (no TZ variable set) I see those displayed seemingly using the Media/Track/_ Create Date. Which seems to be some ‘arbitrary iPhone stuff’… (I mean those do not correlate with the original time in any static - offset manner…)
Metadata 3530: (3531 in next post. EDIT: no I can’t reply anymore, new users can only post so much consecutively. But it shows in the creation_time field the same value as from one of the Create Date fields of the exif data)

So the bottom line, which would resolve it for me on the iphone footage case, use the ‘Date/Time Original’.
And for the GoPro case, I’m not familiar with what ShotCut actually uses/parses. It is somewhere respecting the TZ variable for display purposes. Maybe it could just plainly use the creation_time string in the Date column rather than parsing and respecting TZ. Those are probably not the best fixes and I don’t know how other files from other devices in other timezones behave, and how a general fix would look like.

[brian]

All this inconsistency is why I provided multiple options in the “Set Creation Time” menu option:

With all the different timestamps that can be correctly and incorrectly associated with a file, I don’t think that Shotcut will always be able to pick the timestamp that a user wants. In some cases, you have no choice but to use the “Set Creation Time” action to set it to what you need.

[DDer_Joel]

Yes, good point. But unfortunately, I can’t find the correct date (2024-04-14 9:47:33 and 2024-04-14 14:39:32) in that dropdown of neither of my clips.

I mean, I am aware that I can pick and choose below the exact time, but is not practical, at least for my use case, where I have hundreds of clips, that are all +12h, or the iphone clips where I need to check the exif ‘original date’-field.
And it seems like I’d need to do that individually for each clip.

I mean all I’d wish for would be the ‘Date’ column to be adjustable on which field it displays (and sorts!) from the exif data.
Would It be possible to give the user the option to prioritse on that?
I could imagine a Playlist-specific setting in shotcut, that lets the user change a list.
In my case, I’d put ‘Original Date/Time’ as my first priority (to fix iphone clips) and if that’s not available, plainly display the creation_time value.

(also while we’re at it, I’d like to re-order or hide those columns, because in/start is not important for me, and duration should be the last column, but I think this is off topic here)

I’m happy to help coding/testing if man-power is required.

[DDer_Joel]

Hey everyone stumbling over this. (Sorry for being off-topic a bit)

I’ve tried using ‘exiftool’ in Shotcut to print the DateTimeOriginal form iPhone files instead of MediaCreateDate, even with the exiftool-C-bindings. I’ve had trouble getting it stable, so here is what I am using now:
I created a script that renames my media and then I import all the files into Shotcut and ‘sort by name’ instead of any date.

The script renames all the GoPro/iPhone media files and prepends the filename with the date, such as 20240812-144240+GX015857.MP4 for GoPro files and 20240806-210129+IMG_4975.MOV for iphone files (JPEG,JPG,MP4,PNG,MOV)

The script prefers using DateTimeOriginal and falls back to MediaCreateDate for iPhone files.
(The script also moves all the THMs into a folder, so they are not distracting when importing to Shotcut).
The script also moved the Low-Res files accordingly, such that you can still use Shotcut’s Proxys

This way, I have a solid and coherent chronological order.
I’ve been using this for a while now and I’m quite happy.

Note that you need to have exiftool installed.

[daniel47]

I also ended up making a script using exiftool for this “issue” but it’s troublesome to do this renaming for everything as each device uses its own naming format. Last time I had a time issue I just manually sorted them.

For a while I manually set gopro’s time to UTC and everything was great but when I connect it to my phone it automatically syncs the local time so that didn’t last long.