Introducing the "Reduce Noise" filter


#1

Version 16.04 introduces the “Reduce Noise” filter. This filter can be useful for reducing noise or film grain in a source clip. The filter works by creating a Gaussian blur of each frame and then replacing pixels in the original frame with the blurred pixels. Configuration parameters are provided for the radius and strength of the blur. Another configuration parameter defines the threshold to determine if a pixel should be replaced by its blurred version. If the difference between the original pixel and the blurred pixel is below the threshold, then the pixel will be replaced. This feature allows the user to decide how much edge detail should be preserved. A lower threshold value will only blur the flat areas of the image. A larger threshold value will blur then entire image.


#2

I have been learning this filter and experimenting with it yesterday for 2+ hours. I want to share my (hopefully constructive) critique and a suggestion here.

CRITIQUE:
I honestly think this plugin at its current state is useful in a limited context. Depending on the footage treated, it may produce visible improvement in quality on source footage with certain characteristics, but lot of times in my testing it was not producing satisfactory outcome, and I would eventually end up not using it. I do not have image processing expertise in the area of noise reduction, so I can’t tell if the very algorithm used in this plugin has a potential for becoming better with more fine-tuning or not. Currently I will probably not consider using the plugin on most of my footage, except in perhaps special scenarios. I will be looking forward to updates in the future versions of this plugin. But for now I decided to stick to my other workflow for noise reduction with a 3rd party tool. Please see my next paragraph.

SUGGESTION:
I have been working with a 3rd party tool for noise reduction, that produces incredibly good results, fabulously good actually. I would like to share it here, because perhaps your development team (both R&D and the bizdev) could find a way to integrate this. The tool is called Noiseware, and is available both commercially as well as for free as Noiseware Community Edition, which as far as I remember is also open source. I am using the community edition, on Linux via wine. Except for video production, where I started using the commercial edition because of batch processing. I can export video clip as sequence of frames, batch process them with Noiseware, then import back into a video clip. This process is slow, and incorporates extra steps with 3rd party software included, but the outcome is absolutely worth all the hassle - the resulting denoised clips are just plain beautiful. Because of the community edition, which I think is based on open source libraries, I would like to suggest you consider this for bringing a truly spectacular noise reduction capabilities to Shotcut.
Attached please see a side-by-side comparison of a frame of footage before and after noise removal with Noiseware. The source frame comes from your tutorial on video restoration, so that you can compare the outcome to what you can achieve using your current filter:


*Btw, while Noiseware allows for manual fine-tuning of noise reduction profile, the result included came out of a built-in preset. I just use a preset in my workflow, and I have never been let down by the results.


Reduce Noise - replace smartblur with hqdn3d or nlmeans
#3

Thanks for your feedback.

I was not aware of Noiseware before you mentioned it. The results you posted to look compelling. Unfortunately, I can’t find any source code releases for Noiseware. It appears to be a “pay only” product and the community edition seems to be very old. If you know where I can get access to the source, please do share.

Regarding the current “Reduce Noise” filter, it is currently using the smartblur filter from ffmpeg.
Documentation:
https://ffmpeg.org/ffmpeg-filters.html#smartblur-1
Source:

The algorithm itself is the same one used in the “Smart Blur” filter in Gimp. There is a video tutorial here that describes the options:

I agree that the filter is not perfect. But I do think that it can be effective in reducing grain and noise in most content. I choose it because, at the time, I found it to be the most effective free and open source filter that is able to run in real time (I found some other filters that were more effective, but could not be run in real time).

I will keep an eye out for any new open source noise reduction filters that come along. If you see any, please let me know.


#4

I do admit that I was not sure about the open source part, just I thought it was. But I will try to find out. Because the Community Edition, despite being old, does great job. The result I uploaded came from the Community Edition.

I guess the key aspect of the critique I gave is down to the point you made: “…that is able to run in real time (I found some other filters that were more effective, but could not be run in real time)” - this point I find very fair. Noiseware definitely is not at the “real time” level, yet it is still pretty fast. Despite my earlier critique, under this argument I do now see where this filter comes from, and I guess I am more appreciative toward it as well.

Now while I am trying to find anything on the source openness (hopefully) of the Noiseware Community Edition, perhaps an intermediate solution - perhaps a plugin could be considered that could delegate the process (after “Analyze” button like in some other filters) to locally installed Noiseware application provided by the user. Depending on a license perhaps the Community Edition could even be shipped with, but if a user had professional edition, the advantage could be taken of the batch mode. Even if this option does seem a bit overkill at first, the results can potentially be soo good, that it might just be worth a shot! After all, I feel it totally pays off to go about the process in a totally manual DIY fashion. It is just so worth seeing the outcome.


#5

Btw, if only you are interested, I could even contribute by trying to write a script for calling Noiseware CE for Windows, Linux and Mac (on Linux and Mac it would depend on wine). You just tell me what calling format you’d like for my helper program, how would you provide frames to me as input. And I can handle the rest.

For Win and Mac plugins are available for some host applications, so perhaps Shotcut could use that (user would provide plugin, SHotcut would provide interface to use it). Linux isn’t covered. I could do what I described.


#6

Thanks for the offer. However, I predict that a plugin to script out to a closed source application in wine is unlikely to be accepted by the project leaders. My suggestion would be to find the best free/open source denoise implementation out there and then see if we can get that integrated somehow. For example, open the Gimp and find out which filters work best in there. Or look at RawTherapee and see what they have available. There are a lot of noise reduction implementations scattered across various open source projects.


#7

Sorry for English, translated by Google translator.
I found for myself a simple super method of cleaning video and pointing the difference(sharpening) through ffmpeg, here is the line for the terminal

ffmpeg -i imput.mp4 -vf edgedetect=low=0.1:high=0.4,hqdn3d=4.0:3.0:6.0:4.5,nlmeans=10:11:1 -vf unsharp=7:7:3:7:7:3,dctdnoiz=4.5,pp=tn/hb/vb -vf unsharp=7:7:3:7:7:3,hqdn3d=4.0:3.0:6.0:4.5,pp=tn/hb/vb/dr -vf dctdnoiz=4.5,unsharp=5:5:1.1:5:5:1.1,pp=tn/hb/vb/ha/va/h1/v1/dr/al output.mp4

This is how the three filters are then three and three and the fourth time three filters in one line.
By coding on my dual-core celeron, the video minute at 720 takes somewhere ~40 minutes. This is better than sharpening and removing noise filters in Shotcut