Digikam/SoK2012/AutoNR: Difference between revisions

From KDE Community Wiki
m (add Mentoring category)
 
(50 intermediate revisions by 2 users not shown)
Line 1: Line 1:
[[Category:Mentoring]]
= digiKam SoK 2012 Auto Noise Reduction =
= digiKam SoK 2012 Auto Noise Reduction =


digiKam already has a good noise reduction algorithm based on Wavelets theory working in YCrCb color space, but it is with manual NR settings. What we need is a Automatic NR settings adjustment option depending upon the input image's noise level and the type of noise present. This would automate the task and will also make it easier for newbies to find the NR settings which 'just works' for the input image.
== Sypnosis ==
 
digiKam already has a good electronics noise [5] reduction algorithm [3] based on Wavelets theory [6] working in YCrCb color-space [4]. It based on a fork of original implementation of Gimp Denoise tool [1], including several improvements to support 16 bits color depth image and memory leak/bugfixes. Original implementation provides more experimental implementation to support RGB and CIELAB color-spaces to process noise, which have been dropped in digiKam implementation. Only Luminance and Chrominance settings from YCrCB color-space have been preserved because it's the most common way to process electronics noise in digital imagery.
 
The Wavelets Algorithm implemented in Gimp Denoise Tool is based on DCraw [9] Noise Cleaner code and later ported to Ufraw [10]. The original code from DCraw use an "À Trous" Discrete Wavelet Transform described into "The Handbook of Astronomical Image Processing" book from Richard Berry and James Burnell, chapter 18 [11]. Gimp NR tool author has also re-used this theory to improve original code.


http://farm9.staticflickr.com/8452/7998679054_5d334b51cd_d.jpg
http://farm9.staticflickr.com/8452/7998679054_5d334b51cd_d.jpg
[http://www.flickr.com/photos/digikam/7998679054/sizes/o/in/photostream/ full size screenshot from Flickr]
digiKam NR tool use manual NR settings. What we need is a Automatic NR settings adjustment option depending upon the input image's noise level and the type of noise present. This would automate the task and will also make it easier for newbies to find the NR settings which 'just works' for the input image.
Closed source program provide a way to compute automatically NR settings accordingly with image contents and camera capture settings, as Sony RAW Image Converter program from Microsoft Windows [7].
== Denoise Gimp Tool ==
The wavelet denoise plugin for The GIMP is an algorithm copied and slightly altered from the UFRaw program (which inherited the algorithm from dcraw). Instead of denoising all RGB channels at once the plugin implementation allows to denoise the RGB channels individually and - even more useful - to denoise the YCbCr or CIELAB channels individually. The colour model conversions are nearly lossless as the internal calculations are done in floating point numbers and rounding errors are avoided.
For camera pictures we advise to use YCbCr mode (which is nearly lossless) and to at least denoise the Cb and Cr channels to reduce the chroma (colour) noise. Cameras shooting in JPEG mode normally have chroma noise already reduced. If desired, the luminance noise can be reduced. This channel usually contains most of the fine structures in the image and should mostly be left alone.
All channels are handled in the same way (regardless of colour space) but independently. One difference to UFRaw and DCRaw algorithm is that It don't use predefined (gaussian) noise level coefficients for each wavelet level, but that It estimate the level by calculating the RMS (Root Mean Square) [12], neglecting image detail. It also use a different "thresholding" function where it specified the "softness" of the threshold.
http://registry.gimp.org/files/a.png http://registry.gimp.org/files/b.png
== General goals ==


Noise level and its type in a image may depend upon various factors such as Camera, ISO and other conditions. A statistical variance between neighboring pixel may give us a rough estimate about the salt & pepper noises. Moreover, this project needs to tested it over a large set of data inorder to get a good relation between detected parameters and settings of wavelet NR.
Noise level and its type in a image may depend upon various factors such as Camera, ISO and other conditions. A statistical variance between neighboring pixel may give us a rough estimate about the salt & pepper noises. Moreover, this project needs to tested it over a large set of data inorder to get a good relation between detected parameters and settings of wavelet NR.


A simple workflow : from an Input Image we will detect noise level and type of noise; use other info
A simple workflow : from an Input Image we will detect noise level and type of noise; use other info available like camera model, ISO, etc; and to complete, compute Auto Adjust wavelets NR settings.
available like camera model, ISO, etc; and to complete, compute Auto Adjust wavelets NR settings.


One of the primary goal is to implement a Noise Level Function (NFL) which works using using a single image[1]. Here in the image data is fit in each region with a smoothing function, then noise is estimated according to the residue remaining. This may over estimate some images which naturally have frequent changes in color, textures, lighting, etc.  
One of the primary goal is to implement a Noise Level Function (NFL) which works using using a single image [2]. Here in the image data is fit in each region with a smoothing function, then noise is estimated according to the residue remaining. This may over estimate some images which naturally have frequent changes in color, textures, lighting, etc.  


Secondly we need to map the obtained Noise Level to the already present NR settings (This would certainly require much of testing with the real image dataset).
Secondly we need to map the obtained Noise Level to the already present NR settings (This would certainly require much of testing with the real image dataset).
Due of lots of different types of electronics noise generated by digital still camera, a collection of noisy images must be collected. A command line test program must be written to check implementation outside digiKam GUI. Some sample files can be already found in digiKam test collection [8].


= Project Timeline =
= Project Timeline =


== Updates ==
== Updates ==
The working model of the nrestimate function with a bit of gui has been posted in https://github.com/Kenzo450D/estimate-noise-image


== Todo ==
== Todo ==
Line 29: Line 56:
|-
|-
| NLF
| NLF
| Implement the function for Noise estimation as in [1]
| Implement the function for Noise estimation as in [2]
| Aniket
| Sayantan Datta
| 1 June
|  
|
| Completed
|-
| ANR
| Adapt the noise levels to the parameters of NR engine
| Sayantan Datta
|
| Completed
|-
| CDS
| Change the code to digiKam style coding
| Sayantan Datta
|
| Completed
|-
|
| Create the files for /core/libs/dimg/filters/nr/nrestimate.xxx
| Sayantan Datta
|
| Completed
|-
|
| Create and test CUI version in /core/tests for stability and bugs
| Sayantan Datta
|
| Completed
|-
|
| Final changes in GUI files
| Sayantan Datta
| 31-oct-2012
| Completed
|-
|
| Documentation
| Sayantan Datta
|
| Under Progress
|}
|}


==Reference and Links==
==Reference and Links==


[http://people.csail.mit.edu/celiu/denoise/estnoise/ Noise Estimation from a Single Image] : Richard Szeliski; Sing Bing Kang; Ce Liu William; T. Freeman, 2006 [1]
[http://registry.gimp.org/node/4235/ Original Gimp Wavelets NR tool] [1]


[http://people.csail.mit.edu/celiu/denoise/estnoise/ Noise Estimation from a Single Image] : Richard Szeliski; Sing Bing Kang; Ce Liu William; T. Freeman, 2006 [2]


[http://iplab.dmi.unict.it/download/Elenco%20Pubblicazioni%20(PDF)/International%20Conferences/5.3-1%20-%20ICCE%202005.pdf Fast method for noise level estimation and integrated noise reduction] : Bruna, A.;  Messina, G.;  Spampinato, G., 2005
[http://iplab.dmi.unict.it/download/Elenco%20Pubblicazioni%20(PDF)/International%20Conferences/5.3-1%20-%20ICCE%202005.pdf Fast method for noise level estimation and integrated noise reduction] : Bruna, A.;  Messina, G.;  Spampinato, G., 2005


[https://bugs.kde.org/show_bug.cgi?id=166820 Bugzilla entry for this project]
[https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/show/libs/dimg/filters/nr Current digiKam NR Wavelets filter and Settings widget source code] [3]
 
[https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/imageplugins/enhance/noisereductiontool.cpp Current digiKam NR tool for Image Editor source code]
 
[http://en.wikipedia.org/wiki/YCbCr YCbCr Color Space definition from Wikipedia] [4]
 
[http://en.wikipedia.org/wiki/Image_noise Electronics Noise definition from Wikipedia] [5]
 
[http://en.wikipedia.org/wiki/Wavelet Wavelets Theory definition from Wikipedia] [6]
 
[http://farm8.staticflickr.com/7060/6788459214_c51035abf7_o_d.png digiKam vs Sony Image Converter NR tool] [7]
 
[http://digikam3rdparty.free.fr/TEST_IMAGES/NOISE/ NR images collection] [8]
 
[http://www.cybercom.net/~dcoffin/dcraw/ DCraw project] [9]


[https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/show/libs/dimg/filters/nr Current digiKam NR Wavelets filter and Settings widget source code]
[http://ufraw.sourceforge.net/ UFraw project] [10]


[https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/imageplugins/enhance/noisereductiontool.cpp Current digiKam NR tool for Image Editor source code]
[http://digikam3rdparty.free.fr/Books/handbook-of-astronomical-image-processing.18-wavelets.pdf Chapter 18 from the book used by DCRaw author to write original de-noise wavelets algorithm] [11]


[http://en.wikipedia.org/wiki/YCbCr YCbCr Color Space on Wikipedia]
[http://en.wikipedia.org/wiki/Root_mean_square RMS - Root Mean Square definition from wikipedia] [12]


[http://en.wikipedia.org/wiki/Image_noise Electronics Noise  on Wikipedia]
[http://pixinsight.com/doc/legacy/LE/20_wavelets/a_trous_wavelet_transform/a_trous_wavelet_transform.html The "À Trous" Discrete Wavelet Transform explained]


[http://en.wikipedia.org/wiki/Wavelet Wavelets Theory on Wikipedia]
[https://bugs.kde.org/show_bug.cgi?id=166820 Bugzilla entry for this project]


{{Note|Please use the talk page to discuss this proposal.}}
{{Note|Please use the talk page to discuss this proposal.}}

Latest revision as of 10:25, 31 March 2013


digiKam SoK 2012 Auto Noise Reduction

Sypnosis

digiKam already has a good electronics noise [5] reduction algorithm [3] based on Wavelets theory [6] working in YCrCb color-space [4]. It based on a fork of original implementation of Gimp Denoise tool [1], including several improvements to support 16 bits color depth image and memory leak/bugfixes. Original implementation provides more experimental implementation to support RGB and CIELAB color-spaces to process noise, which have been dropped in digiKam implementation. Only Luminance and Chrominance settings from YCrCB color-space have been preserved because it's the most common way to process electronics noise in digital imagery.

The Wavelets Algorithm implemented in Gimp Denoise Tool is based on DCraw [9] Noise Cleaner code and later ported to Ufraw [10]. The original code from DCraw use an "À Trous" Discrete Wavelet Transform described into "The Handbook of Astronomical Image Processing" book from Richard Berry and James Burnell, chapter 18 [11]. Gimp NR tool author has also re-used this theory to improve original code.

7998679054_5d334b51cd_d.jpg full size screenshot from Flickr

digiKam NR tool use manual NR settings. What we need is a Automatic NR settings adjustment option depending upon the input image's noise level and the type of noise present. This would automate the task and will also make it easier for newbies to find the NR settings which 'just works' for the input image.

Closed source program provide a way to compute automatically NR settings accordingly with image contents and camera capture settings, as Sony RAW Image Converter program from Microsoft Windows [7].

Denoise Gimp Tool

The wavelet denoise plugin for The GIMP is an algorithm copied and slightly altered from the UFRaw program (which inherited the algorithm from dcraw). Instead of denoising all RGB channels at once the plugin implementation allows to denoise the RGB channels individually and - even more useful - to denoise the YCbCr or CIELAB channels individually. The colour model conversions are nearly lossless as the internal calculations are done in floating point numbers and rounding errors are avoided.

For camera pictures we advise to use YCbCr mode (which is nearly lossless) and to at least denoise the Cb and Cr channels to reduce the chroma (colour) noise. Cameras shooting in JPEG mode normally have chroma noise already reduced. If desired, the luminance noise can be reduced. This channel usually contains most of the fine structures in the image and should mostly be left alone.

All channels are handled in the same way (regardless of colour space) but independently. One difference to UFRaw and DCRaw algorithm is that It don't use predefined (gaussian) noise level coefficients for each wavelet level, but that It estimate the level by calculating the RMS (Root Mean Square) [12], neglecting image detail. It also use a different "thresholding" function where it specified the "softness" of the threshold.

a.png b.png

General goals

Noise level and its type in a image may depend upon various factors such as Camera, ISO and other conditions. A statistical variance between neighboring pixel may give us a rough estimate about the salt & pepper noises. Moreover, this project needs to tested it over a large set of data inorder to get a good relation between detected parameters and settings of wavelet NR.

A simple workflow : from an Input Image we will detect noise level and type of noise; use other info available like camera model, ISO, etc; and to complete, compute Auto Adjust wavelets NR settings.

One of the primary goal is to implement a Noise Level Function (NFL) which works using using a single image [2]. Here in the image data is fit in each region with a smoothing function, then noise is estimated according to the residue remaining. This may over estimate some images which naturally have frequent changes in color, textures, lighting, etc.

Secondly we need to map the obtained Noise Level to the already present NR settings (This would certainly require much of testing with the real image dataset).

Due of lots of different types of electronics noise generated by digital still camera, a collection of noisy images must be collected. A command line test program must be written to check implementation outside digiKam GUI. Some sample files can be already found in digiKam test collection [8].

Project Timeline

Updates

The working model of the nrestimate function with a bit of gui has been posted in https://github.com/Kenzo450D/estimate-noise-image

Todo

Milestone name Milestone description Assigned to Expected Start Date Status
NLF Implement the function for Noise estimation as in [2] Sayantan Datta Completed
ANR Adapt the noise levels to the parameters of NR engine Sayantan Datta Completed
CDS Change the code to digiKam style coding Sayantan Datta Completed
Create the files for /core/libs/dimg/filters/nr/nrestimate.xxx Sayantan Datta Completed
Create and test CUI version in /core/tests for stability and bugs Sayantan Datta Completed
Final changes in GUI files Sayantan Datta 31-oct-2012 Completed
Documentation Sayantan Datta Under Progress

Reference and Links

Original Gimp Wavelets NR tool [1]

Noise Estimation from a Single Image : Richard Szeliski; Sing Bing Kang; Ce Liu William; T. Freeman, 2006 [2]

Fast method for noise level estimation and integrated noise reduction : Bruna, A.; Messina, G.; Spampinato, G., 2005

Current digiKam NR Wavelets filter and Settings widget source code [3]

Current digiKam NR tool for Image Editor source code

YCbCr Color Space definition from Wikipedia [4]

Electronics Noise definition from Wikipedia [5]

Wavelets Theory definition from Wikipedia [6]

digiKam vs Sony Image Converter NR tool [7]

NR images collection [8]

DCraw project [9]

UFraw project [10]

Chapter 18 from the book used by DCRaw author to write original de-noise wavelets algorithm [11]

RMS - Root Mean Square definition from wikipedia [12]

The "À Trous" Discrete Wavelet Transform explained

Bugzilla entry for this project

Note

Please use the talk page to discuss this proposal.