TMCnet News

Under-Exposed Image Enhancement Based on Relaxed Luminance Optimization [Sensors & Transducers (Canada)]
[April 22, 2014]

Under-Exposed Image Enhancement Based on Relaxed Luminance Optimization [Sensors & Transducers (Canada)]


(Sensors & Transducers (Canada) Via Acquire Media NewsEdge) Abstract: In order to overcome the dimming and blurring effect resulted from deficient enhancement or the haloing and noising problem caused by excessive enhancement, this paper proposes a luminance optimization based under-exposed image clearness enhancement algorithm, which treats it as the simultaneous augmentation of luminance and contrast, and combines them in an optimization framework under the expectations of augmented luminance and gradient fields. We adopt a relaxation strategy in the gradient energy terms of the framework to further suppress the noise amplification phenomenon. Experimental results demonstrate that our algorithm is simple and flexible, and obtains clear pictures for both locally and globally under-exposed images at the cost of only solving a system of sparse linear equations. Copyright © 2013 IFSA.



Keywords: Under-exposed image enhancement, Luminance optimization, Relaxation strategy.

(ProQuest: ... denotes formulae omitted.) 1. Introduction Bad environment is unavoidable when we take a photography, which usually leads to the exposure deficiency or excess phenomenon existing in the acquired frames. It makes us difficult to see the visual information hidden in the darkness or highlights of the pictures, which not only greatly reduce their appreciable aesthetic feeling, but also brings immeasurable loss to such important application fields as visual surveillance, judicial forensic, etc. Therefore, we must enhance the clearness of the images. However, this paper is particularly concerned about the under-exposed image clearness enhancement, which seeks to present the visual content in the under-exposed image clearly, especially making it easy for us to identify the detail information contained in the under-exposed area.


2. Related Works Image clearness enhancement belongs to the category of image enhancement [1]. The conventional methods, such as histogram equalization and Gamma correction, augment the global image contrast by nonlinear luminance stretching on the logarithm domain. However, they are not convenient to be used, because they require relatively complex parameter settings or user interaction. In addition, there are some constraints among the different regions in an image, hence global mapping functions often sacrifice the image contrast in the highlight area to enhance those in the low-light area, which leads to the reduced contrast range in the fully exposed regions and the loss of the original image luminance hierarchy.

High dynamic range (HDR) image processing shares some similarities with the under-exposed image enhancement because a HDR image can be linearly transformed to be an ordinary low dynamic range (LDR) image, which is similar to an underexposed image. Tumblin et al. [2] put forward the tone mapping problem at first, namely how to display HDR images in a relatively small dynamic range devices. According to the sensitivity of human eyes to luminance and contrast, they propose a global mapping function. Then, Larson et al. [3] proposed an improved histogram customized method, which can fully utilize the dynamic range of luminance and avoid the flat regions to be pulled. Dargo et al. [4] presented an adaptive log mapping method, which can easily and effectively perform tone mapping with a global operator. Lischinski et al. [5] brought forth an interactive local tonal adjustment algorithm. Users simply paint a few strokes with the brush in the area to be adjusted, and set such parameters as luminance and contrast by the slider. The algorithm then propagates these parameters to the entire region by minimizing the edge-preserving energy.

Retinex theory [6] is a model based on how human visual system perceives the color and luminance of an object. The basic idea is that the perceived illumination of a point is not only related to its absolute illumination value, but also influenced by the surrounding illumination values. It can be used to compress the dynamic range of the image, improve the image contrast, and show details drown in the shaded area effectively. The general procedures of the Retinex method can be summarized as follows: first to estimate the illumination from the original image, then to remove the illumination in logarithmic domain from the original image, so as to return the enhanced image. However, with the assumption of gradual illumination changes which is usually inconsistent with the actual situations, traditional Retinex method has a distinct drawback, i.e., it produces haloing effects in the area with strong image contrast. Inspired by Retinex theory, Choudhury et al. [7] proposed a perception-driven automatic contrast enhancement algorithm. They first extract the reflection coefficients and the illumination information in the image based on the color constancy hypothesis; then they multiply the adaptive augmented illumination information by the reflection coefficients to obtain the contrast enhanced image. Although it avoids the haloing phenomenon to a certain extent, the noising problem is still obvious.

In recent years, some scholars propose the gradient domain based luminance adjustment methods. For HDR image processing, Fattal et al. [8] constructed an expected image gradient filed based on the specified luminance values in the image with user interaction, and obtained a resulted image by minimizing the difference of its gradient filed with the expected one. Gradient domain based image enhancement algorithms are better in preserving the image details and hierarchy information, but they show such defects as the uncontrollable noising, dimming or blurring effects, which fail to meet the requirements of human perception. Guo et al. [9] brought forward an automatic tone adjustment method for the under-exposed image. They adjusted the tone values of pixels by remapping the dynamic range with a tonal mapping operator, which is composed of a global operator and a local one. The former adjusts the tone values of the under-exposed image with linear transformation and a non-uniform luminance reduction function, and the latter is used for noise suppression and detail enhancement. Although it can get better results through fine parameters setting, the trial process for the optimal parameters is not only heavy and tedious, but also shows higher requirements for the users. Wang et al. [10] devised an automatic luminance adjustment algorithm which is based on gradient domain manipulation. It first partitions the image with luminance clustering, then computes the luminance adjustment operation for each region, finally obtains the enhanced result by solving the gradient constrained equations. However, the overall image enhancement effect looks dimming.

In short, there are various deficiencies in the existing image clearness enhancement methods, such as the dimming and blurring effect resulted from under-enhancement, the haloing and noising effects caused by over-enhancement, and the complexity problems in algorithm, operation, or computation. In order to overcome the above problems, we design a luminance optimization based under-exposed image clearness enhancement framework with relaxation strategy, which is simple, flexible and effective.

3. Under-Exposed Image Enhancement Based on Relaxed Luminance Optimization In order to overcome the dimming and blurring effect or the haloing and noising phenomenon existing in the state-of-the-art methods, we propose a luminance optimization based under-exposed image clearness enhancement algorithm with relaxation strategy. Because human visual system is good at identifying the image with high luminance and high contrast, we treat the under-exposed image clearness enhancement problem as the simultaneous augmentation of luminance and contrast, and combine them in an optimization framework under the expectations of the augmented luminance and gradient fields.

The main procedures of our algorithm are as follows: Firstly, we extract the luminance information L as well as the chromaticity and saturation information ab from the under-exposed image I by transforming it from RGB to Lab color space. Secondly, we preliminarily augment the luminance and contrast of the image respectively, i.e. increasing the luminance from L to V, and boosting the gradient that reflect image contrast from VL to (VL)'. Thirdly, under the constraints of the expected luminance field L' and the expected gradient field (VL)', we get the optimal luminance field L" via energy optimization. Finally, we obtain the clearness enhanced result of the under-exposed image with the optimal luminance field L" and the chromaticity and saturation information ab through color space transformation from Lab to RGB .Fig. 1 shows the flow chart of our luminance optimization based under-exposed image clearness enhancement algorithm.

Let L(p) represent the original luminance value of a pixel p in the under-exposed image I. In order to obtain its optimal luminance value L''(p) , we construct the following energy function under the constraints of the expected augmented luminance and gradient field: ...

where E,(L") is the luminance energy term, Eg(L") is the gradient energy term, \ and À2 are the weight coefficients for the luminance energy term and the gradient energy term respectively, which are used to balance their influences.

For the purpose of increasing the luminance values of the under-exposed image, we try to conform the optimal luminance field L"(p) to the expectation of the augmented luminance field L'(p), and thus define the luminance energy term as follows: ...

where L'(p) denotes the expectation of the augmented luminance value for the pixel p . A rapidly rising enhancement function is adopted to amplify the luminance value of pixel p from L(p) to L'(p), which is defined as follows [11]: (ProQuest: ... denotes obscured text omitted.) with Zmax indicating the maximum luminance value of all pixels in the under-exposed image, ß is the adjustment factor for the augmentation amplitude. As shown in Fig. 2, the bigger ß leads to the larger adjustment amplitude.

With the aim of enlarging the contrast features of the under-exposed image, we try to align the gradient VL"(p) of the optimal luminance field L"(p) with the expectation of the augmented gradient field (VL(p))', and hence give the gradient energy term Eg(L") as follows: ...

where (VL(p))' is the expectation of the augmented gradient value for the pixel p . We use the same rapidly rising function as the luminance enhancement function to enlarge the luminance gradient value of pixel p from VL(p) to (VL(p))' . However, larger gradient change brings about distinct noising and haloing effects in the clearness enhanced version of the under-exposed image. To overcome this drawback, we adopt a relaxation strategy and add a relaxation factor to the gradient energy term, which can weaken the constraint of the gradient energy term for the pixels with sudden gradient change, i.e.

...

hereinto RF(p) is the relaxation factor, a is the relaxation exponent, which takes the range of 1 ~ 5 . It can be categorized into the relaxed optimization schemes when a > 1, or the non-relaxed optimization schemes when a = 0 .

Since the luminance energy term and the gradient energy term are quadratic functions, the whole energy function is quadratic, which is shown as follows: ...

We utilize the linear least square method to minimize this energy function and adopt the GaussSeidel iterative method to solve for the optimal luminance field L"(p).

Our technical framework is flexible. As long as we choose different expectations for the enhanced luminance or gradient field, it outputs different image clearness enhancement effects.

In order to further demonstrate the effectiveness of our choice for the expected luminance and gradient field, we provide two schemes with extreme conditions for comparison, i.e. the gradient enhancement scheme and the luminance enhancement scheme.

For the gradient enhancement scheme, we only augment the expected gradient field and fix the expected luminance field L'=L. With the luminance enhancement scheme, we only enlarge the expected luminance field and set the expected gradient field (V¿)'= VZ . Therefore, the energy functions for above two schemes can be written as follows: ...

4. Experimental Results The experimental platform for our technical framework is a PC with Intel® Core(TM) i5 CPU, 4 GB memory, which is installed with Windows XP operating system. The software prototype for the luminance optimization based under-exposed image clearness enhancement is implemented with Microsoft Visual C++ 8.0 development environment.

Fig. 3, Fig. 4 and Fig. 5 show three underexposed image enhancement results, where (a) is the input image.

As Wang et al result [10] shown in (b), we can observe that the luminance and contrast in the originally under-exposed low-light area have been slightly improved, but the luminance in the original highlight have been greatly reduced, which make it hard to identify the inner detail information, (c) shows the gradient enhancement result which only augments the expected gradient field.

It greatly enhances the detail information by increasing the local contrast, but it holds almost the same luminance information as the input underexposed image, therefore it looks a bit dimming on the whole, (d) shows the luminance enhancement result which only enlarges the expected luminance field.

It greatly augments the luminance information via luminance boosting, but it retains nearly the similar local contrast information with the input underexposed image, consequently it looks a little blurring on the whole.

The non-relaxed optimization result for the underexposed image clearness enhancement is shown as (e). Both luminance and local contrast are greatly enlarged so that it is easy to recognize all the information in the enhanced image.

However, the exceedingly augmented local contrast leads to severe noising and haloing effects, as a result it is necessary to appropriately suppress the excessive contrast augmentation phenomenon, (f) shows the relaxed optimization result for the underexposed image clearness enhancement.

Although its sharpening degree is slightly lower than (e), it eliminates such unfavorable effects as noising and haloing in (e), and achieves clearer effect with simultaneous luminance and contrast augmentation than (b), (c) and (d).

Fig. 6 displays an under-exposed image clearness enhancement example based on the relaxed optimization scheme with different weight coefficients for the gradient energy term ^ \ = 1 ). It is obvious that when X2 is relatively small ( ^ < 1 ), the constraint effect of the gradient energy term is weakened, and the image contrast is not improved to a certain extent, thereby the clearness enhanced image appears a little fuzzy.

With the gradual increment of X2, the constraint effect of the gradient energy term is strengthened, and the detail information in the clearness enhanced image becomes clearer.

However, after ^ is increased to a certain extent ( X2 > 5 ), the excessive constraint of the gradient energy term begins to bring about noise amplification and haloing effects.

The experimental results are consistent with the theoretical analysis for the influence of the weight coefficients, and we generally take Ä, = 1 and 1<^<5.

Fig. 7 presents the under-exposed image clearness enhancement results based on the relaxed optimization scheme with different expected luminance and gradient augmentation parameters. With the gradual increment of the luminance enhancement parameter, the enhanced images in the top row of Fig. 7 change from dark to bright. However, overlarge luminance enhancement parameters ( ß, > 5 ) gather the luminance values of all pixels into the highlight range, and result in a compressed dynamic range for the enhanced image, which is unfavorable to the exhibition of the image details. With the gradual increment the gradient enhancement parameter, the enhanced images in the bottom row of Fig. 7 change from blurring to sharpening. However, overlarge gradient enhancement parameters ( ßg > 5 ) bring about noising and haloing effects. To meet the requirements of the ideal clearness enhancement effect, we usually take 1 < pi,pg < 5 .

5. Conclusions and Future Work This paper proposes a luminance optimization based under-exposed image clearness enhancement algorithm, which can resolve the dimming and blurring effects induced by deficient enhancement or the noising and haloing effects resulted from excessive enhancement. We regard the underexposed image clearness enhancement problem as the simultaneous augmentation of luminance and contrast, and combine them in an energy optimization based technical framework with the constraints of the expectation for the augmented luminance and gradient fields. In addition, we adopt a relaxation strategy and add a relaxation factor to the gradient energy term, which weakens its constraint effect for the pixels with large gradient change and further suppress noise amplification and haloing phenomena. Our algorithm is simple and flexible, and can obtain clear pictures for both locally and globally underexposed images at the cost of only solving a sparse linear equation, which is demonstrated by a lot of experimental results.

Acknowledgements This work is supported by the National Natural Science Foundation of China under Grant No. 61003188, the open funding project of State Key Lab of Virtual Reality Technology and Systems at Beihang University under Grant No. BUAA-VR-13KF-2013, the Zhejiang Provincial Key Laboratory of Electronic Commerce and Logistics Information Technology under Grant No. 2011E10005, the Zhejiang Provincial Commonweal Technology Applied Research Projects of China under Grant No. 2013C33030, the Education Department of Zhejiang Province under Grant No. Y201018011, and the Key Technology Innovation Team Building Program of Zhejiang Province under Grant No. 2012R10041-15.

References [1] . Rafael C. Gonzalez, Richard E. Woods, Digital Image Processing, Third Edition, Prentice Hall, 2010.

[2] . Jack Tumblin, Holly Rushmeier, Tone reproduction for realistic images, IEEE Computer Graphics and Applications, Vol. 13, Issue 6, 1993, pp. 42-48.

[3] . Gregory Ward Larson, Holly Rushmeier, Christine Piatko, A visibility matching tone reproduction operator for high dynamic range scenes, IEEE Transaction on Visualization and Computer Graphics, Vol. 3, Issue 4, 1997, pp. 291-306.

[4] . F. Drago, K. Myszkowski, T. Annen, N. Chiba, Adaptive logarithmic mapping for displaying high contrast scenes, Computer Graphics Forum, Vol. 22, Issue 3,2003, pp. 419-426.

[5] . Dani Lischinski, Zeev Farbman, Matt Uyttendaele, Richard Szeliski, Interactive local adjustment of tonal values, ACM Transactions on Graphics, Vol. 25, Issue 3,2006, pp. 646-653.

[6] . Edwin H. Land, An alternative technique for the computation of designator in the Retinex theory of color vision, Proceedings of the National Academy of Sciences of the United States of America, Vol. 83, Issue 10,1986, pp. 3078-3080.

[7] . Anustup Choudhury, Gérard Mediono, Perceptually motivated automatic color contrast enhancement based on color constancy estimation, EURASIP Journal on Image and Video Processing, Vol. 2010, Article ID 837237,2010, pp. 1-22.

[8] . Raanan Fattal, Dani Lischinski, Michael Werman, Gradient domain high dynamic range compression, ACM Transaction on Graphics, Vol. 21, Issue 3, 2002, pp. 249-256.

[9] . Guo Yanwen, Xu Xiaodong, Tone adjustment of underexposed images using dynamic range remapping, Lecture Notes in Computer Science, Vol. 5353,2008, pp. 515-524.

[10] . Wang Xiang, Guo Yanwen, Du Zhenlong, Wu Gangshan, Zhang Fuyan, Peng Qunsheng. Automatic brightness adjustment for image and video, Acta Electrónica Sínica, Vol. 37, Issue 4A, 2009, pp. 79-86.

[11] . Eric P. Bennett, Leonard McMillan, Video enhancement using per-pixel virtual exposures, ACM Transactions on Graphics, Vol. 24, Issue 3, 2005, pp. 845-852.

1 Chunxiao LIU,2 Feng YANG 1 School of Computer Science & Information Engineering, Zhejiang Gongshang University, Hangzhou, 310018, China College of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China 1 Tel:+86-571-28008289 E-mail: [email protected] Received: 23 September 2013 /Accepted: 22 November 2013 /Published: 30 December 2013 (c) 2013 International Frequency Sensor Association

[ Back To TMCnet.com's Homepage ]