Intermediate Files for Editing

Brilliant post, Austin. Thank you so much for taking the time to explain. My fundamentally practical and logical brain (some call it simple…) suddenly does not seem so at odds with the thinking behind intermediate files (now that I understand the background and use of intermediate files selectively).

(PS. You are spot on re Eyeframe… I first heard about it when trialling Lightworks. The software and MPEG-2 were both recommended by a fellow Lightworks user).

As things stand; I ended up doing a “first cut” of the original footage and exporting the edits to Huffyuv. Shotcut has struggled in the subsequent edit when I tried to pull all the footage together on the timeline. I have resigned myself to working on the project in smaller sections and then stringing the complete story together in a final edit.

FFV1 is mathematically lossless and has been used for archival purposes. a quick look shows that it seems to compress better than huffyuv as well

(note I’m still reading the below pdf and only skimmed it so far)

As @D_S said, FFV1 is a mathematically lossless format designed for archiving.

The short story behind archive formats is that they strive for 1) the highest compression rates possible using lossless algorithms, 2) the highest resiliency possible using internal error correction codes, and 3) the highest longevity possible by thoroughly documenting the format for others to read.

The trade-off for all these goodies is speed. Here are some sample numbers.

File size: In my tests, FFV1 files are on average 14% smaller than Ut Video Median, and 44% smaller than HuffYUV Median.

Encode time: FFV1 averages 1.5x to 2x the transcode time of Ut Video and HuffYUV.

Capabilities: FFV1 supports pretty much every colorspace under the sun.

However, doing ffmpeg -h encoder=ffv1 and looking at the “Threading capabilities” line quickly reveals our first problem… FFV1 only supports slice as opposed to frame. Slice is slow compared to frame, which is why the encoding time is longer.

Secondly, decompressing the video is so CPU intensive that I can’t even play back a 4K 24fps FFV1 video on a 4-core laptop using MPC-BE media player on Windows 10. Playback speed is down to half of real-time speed due to the decoding effort required. Meanwhile, HuffYUV and Ut Video can play back 4K just fine on the same laptop.

And that’s the basic problem with archive formats. They sacrifice speed for compression rates, which makes them bad candidates for an editing environment. You won’t get real-time previews with archive formats.

FFV1 is designed to be your final render format so that every last one of your colors is saved in as little space as possible. FFV1 is like your master disc after a recording session in a music studio. Once you have it, you can transcode off any other copies in any other formats you like with the highest quality possible (and not have to re-open and re-render your original project). This is useful if you want to submit a video to multiple TV stations but they all have different format requirements.

Basically, FFV1 is terrible for intermediates and proxies due to slow performance. It’s ideal as a master or archive format for professional footage. It’s probably overkill for home video. The reality of history is that popular consumer formats tend to stick around as long as archive formats by sheer force of volume. As such, any H.264 home movies you make today will probably still be viewable 30 years from now, and you can save yourself the space of an FFV1 master that may not even play back at full speed today.

If you want to create intermediates directly through the Shotcut interface, I would personally choose the ProRes option to retain the important color information without sacrificing all speed. I would only choose FFV1 if I were documenting archaeological artifacts for the Smithsonian. :slight_smile:


Would you use the “Quality: 60%” that is preselected with the preset, or 100%?

I’d try it at the default, ProRes is designed to be “visually lossless” and there shouldn’t be a visual difference there just a larger file with a longer encode time(that said feel free to try it at 60/70/80 to see if slightly higher than the default is better for your specific application)

Good question, because the GUI isn’t intuitive here. The Quality drop-down is a permanent part of the user interface, meaning it’s visible whether it is relevant to the codec or not.

For instance, HuffYUV and Ut Video are lossless codecs, so there is no concept of Quality at all. Those codecs capture everything by design, but the Quality drop-down is still there even though it does nothing.

Same for ProRes. The Quality drop-down does nothing. Encoding quality is configured by the vprofile option in the Export tab > Other box. The link below is a useful description of the five profiles available. Profiles 0 and 1 are not visually lossless; they are designed to be lower quality for lower disk usage, but still be ProRes format for editors that need it. Profiles 2 through 5 are visually lossless, with 2 being normal, 3 being for the paranoid (you may as well go true lossless at this point), and 4 if you need 4:4:4 colorspace.

1 Like

If you try the specific preset once with 60% and then with 100%, you get two files with very different size and bitrate!

Woah, that’s new. When did that get linked up? My apologies for the outdated information!

So here’s how we figure this out… Apple has a white paper on ProRes.

Page 25 lists the built-in target bitrates needed by each profile to do its job at the quality expected of it for each resolution. The vprofile parameter still determines the profile to use, and any profile less than 2 will not be visually lossless. Technically, 2 is not perfectly visually lossless either, but it’s so close that people use it for production anyway.

After some test renders, I’ve concluded that Quality at 100% will use the full bitrate needed by the profile as specified on Page 25. Quality lower than 100% will lower the bitrate (drastically!), but now the codec isn’t living up to its expectations. In particular, it would lose its status as visually lossless, at which point other codecs may become more competitive options.

In this light, my approach would be to leave Quality at 100% all the time and just change vprofile if I want a smaller file. I’m pretty sure (but haven’t tested) that better results will happen with a codec designed for the lower bitrate rather than asking a higher-quality codec to work at a lower bitrate than it’s designed for.

Thanks for pointing this change out to me!


Considering the enormous size of files that HuffYUV produces, wouldn’t that statement make more sense for HuffYUV than FFV1? I once exported a 3 minute clip with HuffYUV and the file size came out to close to 8 gigs whereas the same clip came out to about 1 gig and change with FFV1. I can only imagine what it would be if I had done a file that was a whole movie with HuffYUV. Unless I am missing something here and seeing as how both HuffYUV and FFV1 are both mathematically lossless, perhaps HuffYUV is more suited for smaller clips and FFV1 better for longer clips when the concern has to do with file size?

Also @Austin what are your thoughts about @chris319 comment about FFV1 and white noise? Would that affect the colors of a video that I am rendering from a FFV1?

Let’s tackle this puzzle in reverse. HuffYUV and FFV1 are both lossless formats. Neither of them produce any effects on the colors because these formats have zero loss of any color information by definition.

When @chris319 talks about white noise, he is referring to encoder efficiency. As in, when playing back white noise encoded with FFV1, it is very computationally expensive (inefficient) to decode it due to its random nature (no repeating patterns to compress), therefore the file size will be larger, and the playback speed may even slow down if the CPU can’t decode fast enough. Meanwhile, HuffYUV retains fast playback regardless of the video content because its compression method is less aggressive.

Now that we know the characteristics of the two formats, let’s see how they perform in an editing workflow.

FFV1 is a smaller file, but it got that way by burning lots of CPU to achieve higher compression. That means it takes a ton of CPU to decompress it too, and its playback speed will be unacceptably slow for playback and editing on the Shotcut timeline.

HuffYUV is a larger file, but it plays back fast and can be smoothly edited on the timeline.

There are two goals for intermediate files:

  1. Be as true to the original as possible.
  2. Make editing faster than the original.

If you’re trying to achieve both goals with a single intermediate file and format, then HuffYUV is your choice (so far) because it is both true and fast. FFV1 is true, but not fast.

The proxy workflow allows you to split these goals up. The intermediate file does not have to be fast because it isn’t used for editing. So using small-but-slow FFV1 is fine. Only your proxy has to be fast, and it can use HuffYUV.

But these are not the only options, hence the context to my statement about the Smithsonian.

My point was that creating lossless intermediates for anything other than Smithsonian artifacts would be overkill. You won’t be able to see the difference, so why burn the disk? This is why ProRes is popular in studios, because it makes smaller files with visually lossless results and faster edits. So my quote is in the context of creating intermediate files directly from the Shotcut interface (and no proxy workflow), where one must choose between ProRes and FFV1. Given those options, I would go ProRes. There are disk savings to be found in humbly admitting that most of our work produced on consumer hardware will not be archived by the Smithsonian for generations to study after us. Perfection is an expensive vanity that 99.9% of our audiences will never notice. :slight_smile:

If you’re more adventurous and do your own transcodes from the ffmpeg command line, then you have yet another option called Ut Video Median which gets you 30% space savings over HuffYUV while retaining the same playback speed. And it’s lossless like FFV1. This is my format of choice for fast intermediates (as opposed to slow intermediates combined with a fast proxy).

To answer your question about suitability based on length of the clip, ideally that should not be a consideration. These formats are chosen because of color requirements, not because of disk space availability. If you have requirements for absolute 100% color fidelity even into the parts of the spectrum that human eyes cannot perceive, then a lossless format is your only option and you give it whatever disk space it demands. If that’s too high a price to pay, then ProRes was invented for a reason and it will serve you well. :slight_smile:

FFV1, as stated above, was designed for mastering and archiving at the end of the workflow, not for real-time playback during editing. If your computer can play it back real-time, then more power to you. The compromises described above do not apply to you. For most people, they can’t achieve real-time editing on FFV1 and have to use another format like ProRes, HuffYUV, or Ut Video.

Short version: FFV1 and HuffYUV and Ut Video are all lossless formats. If you’re using proxies, you choose the format that has the highest compression to save disk (FFV1). If you’re not using proxies, you choose one based on fast playback for smooth editing (HuffYUV or Ut Video). If you’re doing everything from the Shotcut interface, HuffYUV is easier to export to. If you’re doing command-line ffmpeg, Ut Video will get you some space savings. But, if you are content with a visually lossless codec (which is good enough for 99.999% of the material out there), then ProRes gets you both disk space savings and decent editing speed.

Everything is a compromise (especially time in production) because humans are only impressed by things that are difficult to achieve. :slight_smile: So your challenge is figuring out where you are willing to make those compromises, and you have options for every stage of the workflow. It makes things confusing, but also powerful once understood.

1 Like

Since my previous post I have done some testing on huffyuv and ffv1. I’m seeing some color shifts which I don’t like. For example, given an input of R = 16, G = 180, B = 16, by actual measurement huffvuv and ffv1 are giving me:

R = 18, G = 149, B = 25.

To the naked eye the huffyuv and ffv1 look darker due to the reduced green conent and I do not consider this acceptable. OTOH, I get

R = 17, G = 181, B = 17

Using x.264 with crf 0. Due to rounding error there will always be 2 - 3 points of error so I consider this acceptable.

“Mathematically lossless” is well and fine but if the colors are shifted that drastically it’s a deal breaker in my book. I guess x.264 with crf0 is considered quasi-lossless?

I’ve been using ffmpeg on a command line. Give me a while and I can test it with the Shotcut presets.

ProRes gives:

R 28 -G 171 - B 28

ShotCut’s H.264 gives:

R 28 -G171 -B26. I get the same results with ShotCut’s HEVC and MPEG-2

What are you using to encode it? I haven’t seen any color shifts on HuffYUV myself but it’s possible you’re shifting color during the RGB<>YUV conversion that huffyuv uses.

The accurate x.264 colors are obtained by using ffmpeg directly from the command prompt with no Shotcut involved. The rest are obtained with Shotcut’s presets.

For clarity, I am starting with a .bmp image and encoding it to huffyuv, ffv1, etc.

It looks like it may be a bmp issue

Sounds like two things in play here. One is RGB vs sRGB to YUV colorspace conversion. The other is MPEG range vs JPEG range video. You’re probably familiar with both issues, but I’ll give a brief explanation for everyone else.

MPEG range is more of a broadcast restriction that says RGB values have to be between 16 and 235. 16 is black and all-235s is white. JPEG range is 0-255. This can be controlled with the ffmpeg -color_range parameter. Converting improperly between them can cause raised or crushed whites/blacks plus color shifts in-between.

The other issue is simple colorspace conversion error. Maybe converting from PNG instead of BMP can be a workaround.

I’ve noticed color shifts exporting from Shotcut that I don’t get using ffmpeg directly. Not sure the issue there either. Will write more if I get a chance to test.

It’s not a bmp issue. The same bmp gives good colors when converted using ffmpeg directly and not going through ShotCut.

I think this is it. I’m not worried about it, though as I can use my direct ffmpeg scripts.

Forgot to follow up on this… In theory, x264 at CRF 0 is true lossless. The rounding error is probably colorspace conversions and there’s not a lot that can be done about that.

There will always be rounding error in calculations involving YUV due to the use of floating-point coefficients. There is YCoCg but it doesn’t subsample well. I don’t think you could do 4:2:2 or 4:2:0 in YCoCg.

I was going to use Huffyuv as my interpositive format until I discovered these color errors, so I’m going with good ol’ x.264 with CRF 0. x.264 is a real workhorse.

I am borrowing an old film term:

Interesting note, in further research I discovered WHY I wasn’t seeing a color shift on my end is that my camera is recording as YUV by default according to mediainfo, interesting discovery to say the least I wonder if it has an option to go between that and rgb internally.