Reduce Noise - replace smartblur with hqdn3d or nlmeans


#1

Any chance of the developers adding hqdn3d or nlmeans video denoise filters? I’m attempting a video restoration and am underwhelmed with the Reduce Noise filter. The output looks like Vaseline-smeared lens softness or ‘70s porn. Smartblur is removing too much detail. To keep the detail, I’m not getting enough noise reduction.

Shotcut Reduce Noise Filter

I’ve had good luck removing noise with Handbrake (nlmeans, but hqdn3d also available) and Avidemux (hqdn3d only – doesn’t keep as much detail as nlmeans). I moved my current video project over to Shotcut because I can’t do a decent white balance or color grading in Avidemux.

Sharpen filter is kind of the opposite issue – it amplifies both detail and noise. Working examples of Msharpen in Avidemux and Unsharp and Lapsharp in Handbrake get better detail without amplifying the noise.

The OP implies it’s using smartblur from ffmpeg. Further down the thread, another poster gives a ffmpeg CLI example using both hqdn3d or nlmeans. If all three filters are available in ffmpeg, would that make implementing nlmeans (or hqdn3d) easier?


#2

Have you seen this tutorial?

Here are a few other discussions on Video Restoration that you might have interest in:

Technical Discussion: Interlacing Revisited
More Film Restoration Tools
Technical Discussion: Frame Rates


#3

Would you have any interest in using ffmpeg to first create lossless intermediate files of your old videos that have nlmeans baked into them? From what I see, nlmeans can be very computationally heavy (read: slow). If you’re like me and make several small test exports to make sure the project looks good before rendering the whole thing, then each test will have to re-compute nlmeans every time. That’s a slow preview cycle. If nlmeans was already done on your sources (intermediates), your previews and final export could be faster. I don’t know if you would be a fan of the extra work, but it could get you a fast nlmeans today rather than wait and hope for an update.

Speaking of which, have you happened to use nlmeans on GoPro footage? Did you find any robust general settings? I haven’t tested this yet but sure sounds useful.


#4

Yes I’ve seen the Film Restoration tutorial. Some neat tricks, but not helpful for reducing noise and preserving detail.

No I did not know about Video Cleaner or film 9. "ook interesting. Thanks.


#5

Yes and no. I don’t like the idea of encoding twice. This is what happens:Multiplicity

Besides the problem of making a copy of a copy, I’d be doing white balance and color grading AFTER denoising, which would amplify noise. I’d prefer to denoise last.


#6

GoPro? Ha? I haven’t worked with HD.

This source was probably a Sony Mavica compact camera from the early 2000s. It’s an .mpg file (not 2 or 4, but one). Handbrake will encode x265, NLMeans Medium preset, UnSharp Light preset at >100 fps on a 5-year old 4-core computer. Definitely not state of the art.

I’ve run some DVD sources (720x480) through handbrake with the same codec and filters and get >20 fps. It can queue jobs like Shotcut, so let it run over night.

I’ve looked at ffmpeg CLI options and see a lot of options for nlmeans. Handbrake just gives presets and tunes. I haven’t played with any of the tunes to see. Light or Ultralight preset might be OK from a “good” DVD source. Medium preset for something that is VGA resolution taken with a low quality sensor. Once or twice I’ve used the strong preset and hit it with Medium UnSharp preset.

Yes when Handbrake added nlmeans, it is slower than hqdn3d, but a couple of test runs I did to compare, nlmeans gave a smaller file size AND looked sharper than hqdn3d. I’m willing to take a speed hit to keep detail.


#7

It would not be hard to add those. I evaluated those and many others before I chose the smartblur filter. The main problem with the other filters is high resource utilization. The ffmpeg filters don’t support multi-threading. And many systems would not be able to achieve near-real-time processing with those. I think people have an expectation that if they apply a filter they will be able to see a live preview in Shotcut before exporting. In my opinion, smartblur gives the best balance of results and live preview.

Have you tried applying the smartblur filter multiple times to the same clip? I had some clips for which it helped to apply the filter twice.


#8

Yeah, since you are grading and other stuff, running the denoise last definitely makes sense. Well, if you need something immediately, there’s still the option of exporting your Shotcut project to a lossless file and using ffmpeg or Handbrake to denoise and create your final H.265 from the lossless file. FFV1 might give you a smaller file faster than H.265 Lossless can at that resolution.

I’m glad you’re getting the frame rates you are with your footage! I will simply say that enthusiasm diminishes when running the same filter chain over 4K GoPro sources. :wink:

GoPros and cell phones get noisy in low light. That’s where my interest in nlmeans comes from, as an improvement over ffdshow. I didn’t know nlmeans existed until you mentioned it, so I have you to thank for the next rabbit hole I go down haha.


#9

This sounds like a programmer has nlmeans running across multiple threads. Pretty good technical discussion about what he did and test metrics.

Option to choose number of threads used by NLMeans filter #835


#10

Handbrake does not use the FFmpeg filter. It has its own implementation.


#11

Tried this twice and was happy with the final Handbrake output

Slow (2 encodes), and FFV1/FLAC files are uuuuge, but it’s temporary and concede that having a lossless intermediate file is a good trade-off.


#12

Glad to hear you found something that works for you!

Since you got me down the rabbit hole of denoisers, I figured I would share my research results with anyone who’s interested.

ffmpeg has six denoisers built-in that I was able to find, which I’ve listed below along with their transcoding speeds on a 1080p source video using a four-core laptop computer. I wrote scripts that used a variety of settings with each denoiser to make sure I was seeing the best each one had to offer.

atadenoise (20 fps) - by averaging pixels across frames, it reduces contrast of noise areas to make them less obvious as opposed to using a specialized algorithm to smooth the noise away; this reduces overall image contrast; filter also darkens the overall output

dctdnoiz (1.6 fps) - creates beautiful detail on a still image, but randomizes the noise across frames so much that it actually makes the noise look worse during playback, plus it darkens the output

nlmeans (0.6 fps) - darkens the output, but sometimes has redeeming qualities (more on this later)

hqdn3d (21 fps) - color neutral which is good, but the output looks smeary to me where it loses a lot of fine detail in hair strands and wood grain

owdenoise (0.3 fps) - color neutral wavelet denoiser with stunningly good results on high-res sources

vaguedenoiser (7.6 fps) - another color neutral wavelet denoiser whose output looks identical to owdenoise, but its processing speed is 25x faster; tried every combination of threshold and nsteps, and found the default settings of 2/6 to consistently produce the closest-to-real-life results

I tested the denoisers on videos I took with my own mirrorless camera, meaning I remember what the scene looked like in real life. In one video, there happened to be a guy in a black business dress shirt made of silk or satin or something with a sheen to it, but the sheen wasn’t coming through due to the noise of the original footage. The wavelet-based denoisers were the only ones to remove and smooth the noise such that the fabric regained the smooth sheen you would expect from silk. To my eye, it bumped up the realism of the video an entire notch to see fabric actually look like fabric. The rest of the frame also dropped to zero dancing noise. It turned the video into a still photograph when nothing was moving. I didn’t realize until this experiment that even a tiny amount of dancing noise can seriously detract from the realism of a video, and that a sense of immersion can be restored by getting rid of it. Obviously, vaguedenoiser is my new weapon of choice.

So, about nlmeans… I found a radical difference between the ffmpeg version and the HandBrake version. I think HandBrake wins on every metric. nlmeans in ffmpeg actually makes video look worse (blockier) if the resolution is 1080p or above, or if the video comes from an excellent camera that has little noise to begin with. nlmeans in ffmpeg also can’t be used as a finishing step because it darkens the output, which destroys any color grading that happened before it. But I found two places where nlmeans in ffmpeg outshined the other ffmpeg denoisers: low-resolution video, and very-high-noise video. nlmeans does great at restoring a VHS capture, which I sense from the author’s web site was one of the original design goals. Secondly, in my tests, nlmeans did better than the other ffmpeg denoisers on high-resolution high-noise videos, which in my case meant a smartphone video in low light using digital zoom. Given these two specialized cases where nlmeans performed well, I could see a workflow where I used nlmeans to create denoised intermediates, then color graded the intermediates to fix the darkened output. Running nlmeans on a noisy source then adding it to the timeline and running vaguedenoiser on the total project did not cause any harm in my tests. But for best results, I think HandBrake is still the way to go where nlmeans is involved.

For my purposes, I think I will stick to vaguedenoiser because it’s beautiful on 1080p and 4K, and it is easily added to my existing ffmpeg filter chain when I do my finishing steps. I don’t have to create an intermediate to pass off to HandBrake this way. However, if I came across a particularly noisy source video, I would probably run it through HandBrake before adding it to my Shotcut project to get the same benefits Andrew noticed.

Good luck to everyone, whatever you use.