TMCnet News
Filters Design for Image based Oil, Watercolor and Cartoon Special Visual Effects Rendering [Sensors & Transducers (Canada)](Sensors & Transducers (Canada) Via Acquire Media NewsEdge) Abstract: Although we are in favor of turning our picture into cool looking effects like oil, watercolor and cartoon paintings, the relevant techniques still too elusive for non-professionals to master. In order to overcome such problems existed in the previous methods as complicated operation and model that do required professional experience, we propose an image filtering based method which can achieve image based non-photorealistic rendering for the amateurs to get the oil, watercolor and cartoon image effects efficiently. By analyzing and summarizing the digital characteristics and difficult technical points for the oil, watercolor and cartoon effects, we design an appropriate special visual effects filter for each of them to make the simulated effects vivid. According to our numerous experiments, the designed filters for oil, watercolor and cartoon effects are easy to use and can generate satisfactory non-photorealistic image rendering results. Copyright © 2013 IFSA. Keywords: Filter design, Special visual effect, Oil painting, Watercolor painting, Cartoon painting. (ProQuest: ... denotes formulae omitted.) 1. Introduction Simulation of the real world is one of the ultimate goals of computer graphics, which has never changed ever since. Nowadays, we can utilize the photorealistic rendering technique to generate digital pictures that are almost the same as the real photos captured with cameras. However, the cloning consistency implied in the computer generated pictures is lack of vivid and realistic sense; as a result it fails to convey the artistic situation which is expected to express by the artists. Different from the traditional photorealistic rendering techniques, the non-photorealistic rendering (NPR) techniques aim at imitating and exhibiting the artistic characteristics of the painting and copying artwork in an image with the digital computer, which is constructed on the basis of the subjective human perception and driven by the human knowledge discipline. As users can outstand or dilute the selected features at their will, this technique has unique advantages and applied prospects in such fields as education and illustrative manuals, etc. [1]. Using the non-photorealistic rendering techniques to create the paintings with the artistic features has become a technical hotspot, which can greatly increase the production of paintings and meet different requirements from users. In the international graphics and vision community, NPR also has become a hot research topic. During the international academic conference every year, a number of highquality non-photorealistic rendering technical papers and results come out. For example, Microsoft and Harvard University have proposed a technical framework for image based non-photorealistic rendering, which brings about a special visual effect filter based on the gradient domain [2], The filter can effectively highlight the image's contours and make its color distribution uniform and compatible. 2. Related Works Nowadays, the non-photorealistic rendering technology has been quite extensively studied. And, the technique to simulate the traditional art works such as watercolor painting [3], oil painting [4], cartoon painting [5] and pencil drawings [6], etc. have achieved great progress. For the stylized rendering of oil paintings, Hertzmann put forward the first rendering algorithm based on the strokes [4]. In the construction of a hierarchical image sequences, though the algorithm can depict the oil painting effect, it merely uses a Gauss filter and does not treat it from the painter's view of image decomposition. It also considers little about the relationship between the primary and secondary art contents in the strokes layout process. Thereafter, in order to improve the surface veracity of oil paintings, he develops an illumination model for the oil painting rendering method on the basis of the above work [7]. Then, he comes up with another improvement idea to make the traditional stroke direction drawing further fine [8]. Firstly, user interactions are required to determine the key drawing area; then an energy function is established and the relaxation iterations are applied to gradually evolve the surroundings of the specified area with more rough drawing styles. However, the method has flaws in computational speed and does require specifying the important region center manually. Because the traditional painting stylized rendering algorithms only consider the stroke directions, Hays et al. [9] train the edge information to obtain their weights and then interpolated on the strong edge weights with the radial basis function (RBF). The stylized simulation of watercolor paintings is first introduced by Small with the cellular automata theory based method [10]. It imitates the interaction of pigment and water to obtain a more realistic watercolor effect. Curtis et al. [3] propose a classic watercolor modeling system based on the previous work. The system can simulate fluid flowing and pigment dispersing through a model with three layers, i.e. shallow water layer, pigment deposition layer and capillary layer. It can generate watercolor stylized paintings in an interactive as well as automatic way. At the same time, a typical non-photorealistic 3D rendering model is also provided. The above drawing process can achieve lifelike effect which is close to the real watercolor paintings, at the expense of computational speed, model complexity and higher professional experience of users. Recently, a new watercolor drawing algorithm based on image processing comes out [11]. Integrating the drawing technique of the wet paintbrush on the wet or drying papers with such techniques as image segmentation, image filtering, image synthesis, etc., the algorithm succeeds in synthesizing many translucent watercolor layers to simulate the watercolor painting effects. However, it is difficult for the algorithm to simulate the natural texture effect produced by the pigment spread in the real watercolor paintings. For the stylized rendering of cartoon paintings, Decaudin [12] proposes a 3D scene based rendering algorithm which can work in an automatic way. It first extracts the scene contours with contour detection on the depth maps. Combining the illumination intensity obtained from the Phong model of OpenGL and the shadow calculated with the shadow map technique, it then handle the tonal information of the rendering objects to get the final result. Later, Claes et al. [13] put forward a rapid cartoon rendering method whose advantage lies in the smooth processing of the boundaries during the colorization process. Lander's method [14] completes the cartoon colorization by the texture mapping technique. Nevertheless, the models in the above algorithms are relatively complex. In short, all of the methods mentioned above hold complex models and require complicated operations, which increase the requirement for professional experience of the users. In contrast, we present a special visual effect rendering framework based on image filtering for the non-professional users, which can automatically produce oil, watercolor and cartoon effect paintings. According to the characteristics of different image filtering methods and stylized paintings, it can reproduce the input images with different styles, i.e. oil, watercolor and cartoon paintings. Analyzing and extracting the special characteristics of oil, watercolor and cartoon paintings respectively, we design appropriate special visual effect image filters to simulate these three kinds of painting styles. Then, we implement them to get the processed images and observe whether they meet our requirements. If the outputs are not satisfied, we can adjust the parameters of the special visual effect filters until an ideal effect is achieved. 3. Special Visual Effect Filter Design for Image Based Oil Painting Rendering Compared with other painting effects, the pigment used in oil paintings can be thickly piled and possesses highly plasticity, which make the oil paintings produced the visual rhythm and strength resonating with human thought and emotion. At the same time, for the oil paintings, the continuous works of strokes and the completion of external outlines construction play a positive influence on the image skin texture. The skin texture of oil paintings is a kind of expressive means that would feast our eyes to some degree by conveying some special feelings to us. What's more, it can reflect the artists' special pursuit of beauty of the period. Excellent oil paintings require not only the expression of shape, color, light and texture, but also the aesthetic feeling of skin texture which is manifested by the painting materials or other supporting materials and reasonably performed by the planar stereo texture [15]. In order to get the oil paintings conveniently, experts put forward the computer simulation technique. However, there is a difficult point existed in the creation process of the NPR oil painting stylized images, that is how to make the reference image layer much closer to the one artists draw with hand. Depicting the scene hierarchies is an important characteristic of the oil paintings. Within each layer, the artists use almost the same color and strokes to create the painting. Therefore, we have to generate a more layering coverage according to the input image, so that the final image can better demonstrate this feature of oil paintings and make the whole image looks more artistic. Our approach of image based oil painting rendering uses the image filtering principle. According to the law of large numbers, we use the blocking effect filter and reset each pixel's value with the one most frequently appeared in its (2N + l)x(2N + l) neighborhood. The diagram for the image based oil painting rendering process is shown as Fig. 1. We shape the detailed implementation steps as follows: 1) Split the input color image into three different channel images, i.e. RGB channel images. 2) Apply the blocking effect filter on the G channel image. For each pixel p , we search its (2N + l)x(2N + l) neighboring pixels for the one p which is occurring most frequently, and reset the G value of p with that of p . 3) Combine the processed G channel image with the original RB channel images to get the final synthesized color image with oil painting style. At step 2), we use a blocking effect filter with a template of (2N + l)x(2N + l) kernel size on the G channel image. Obviously, the templates applied with different kernel sizes will turn out different oil painting rendering results. Through lots of experiments, we find that smaller templates would cause the oil painting effects indistinguishable, while larger ones would lead to the loss of the original texture characteristics. Hence, according to the size of the input image and the weight of oil painting bush, we choose the template with kernel size 5x5(N =2). 4. Special Visual Effect Filter Design for Watercolor Stylized Image Synthesis Watercolor stands for a romantic, pure and free painting language in the world of art, which presents such particular characteristics as good wettability and fluidity. And of course, whether a piece of watercolor work is successful or not is determined by many factors, for example the exquisite degree of the pigment, the fluidity of the water and the roughness of the paper. While the wet brush paints on the dry paper, it would deepen the color of the edges, which is the so-called edge darkening. [16]. It is caused by the pigment diffusion driven by the paper surface tension and the followed water penetration from the wet area to the dry ones. When the wet brush draws on the wet paper, it would generate the beautiful pinnate texture along the water flow, which is caused by many factors, such as the flow patterns with the free diffusion of the pigment, the structure of the paper and the little pressure on it. Since the paper has many small holes and the pigment particles have different sizes, different absorption and diffusion effects on the paper under interaction between the water and the pigment will form different darkening edges and pinnate textures. The irregular movement of water and pigment on the paper brings about rich and colorful pinnate textures. The wettability of water leads to its transparency and brightness. To achieve these artistic effects with the computer is a great challenge. To make the watercolor effect come true, we preprocess the input image with a median filter. The existence of noise in the image generation, transmission and transformation process deteriorate them with degradation phenomena. The median filter is a nonlinear processing method which can remove noise while preserving the image edge under certain conditions. Although the median filter may remove some image details, such as fine lines, comers, etc., we choose it to achieve the watercolor effect because the watercolor painting images prefer the vague contours to the distinct details. For the filtered image, we then apply the edge diffusion filter to enhance the edge information in the filtered image, and finally get the watercolor style image. The diagram for the image based watercolor painting rendering process is shown in Fig. 2. The procedures of our watercolor stylized rendering algorithm can be described as follows. 1) Split the input image into R, G, and B three single channel images. 2) Use the standard median filter to preprocess the three images. The basic principle is to replace the value of a pixel in a digital image with the median value of its neighborhood. Let {xp,p(i,j) e 1} represents the gray value of a pixel p(i,j) in the digital image I, we can define the median filtering with a filtering window A on I as follows: ... (1) ... (2) 3) After image preprocessing, most interference factors for our experimental results have been ruled out. We apply the edge diffusion filter on the three channel images respectively to enhance their edge information weakened by the median filter. In other words, we convolute the edge diffusion template with preprocessed images. Then we reset the value of pixel p as the average value Avg{p) calculated in the template centered on the pixel p . The convoluted image is the expected watercolor stylized image which presents watercolor characteristics. Convoluting with the edge diffusion filters with different 3x3 kernel templates will generate different watercolor effects. The templates with greater absolute weights for the neighboring pixels highlight the image details and lose the vague contours, which is contradictory to the nature of the watercolor paintings. Through lots of experiments, we determine to adopt the following template ... to mimic the hand-painted watercolor effect. 5. Image Based Rendering with Cartoon Style via Special Visual Effect Filtering Cartoon is a popular media with artistic form, whose expression mean relies on the modeling and painting arts. By analyzing various painting styles of the cartoonists, we conclude two main features, i.e. edge lines and colors. Coherent and clear lines not only highlight the leitmotiv and the main point of the paintings, but also express their content clearly [17]. Simple and uniform colors used in the cartoon paintings, do make them vivid and attractive. As a result, the key point is to preserve the contour information as much as possible and simplify the color distributions to be uniform while transforming a real picture to a cartoon effect image. In order to get the cartoon effect image, we adopt the bilateral filter and the gradient domain filter to design our cartoon effect filter. The former can maintain the contour features, while the latter can uniform the color distributions. According to the processing flowchart shown in Fig. 3, our cartoon stylized rendering process can be decomposed into four steps. Firstly, the input color image is converted from the equipment related RGB color space into the device independent Lab color space to broaden the color gamut and make up the uneven color distribution in the RGB color space. Secondly, the bilateral filter is used to remove image noise while preserving the important edges and details of the image. In this way, we reach the purpose of saving the contours of the image. Thirdly, the gradient filter is applied to equalize the color distribution of the image. In detail, we first calculate the gradient G(p) for each pixel p . Let Gmjn and Gmax denote the minimum and maximum cut offs of the pixel gradient respectively, which is specified by the users. We then compare G(p) with Gmax and Gmio as shown in equation (3) and (4) respectively. ... (3) ... (4) Finally, we convert the image from Lab to RGB color space, which is our expected image with cartoon style. Obviously, larger G^ and G^0 are favorable for the stylized cartoon rendering effects. 6. Experimental Results We evaluated our technical framework on a PC with Windows XP operating system and Intel® Core(TM) i5 CPU 650, 3.2 GHz x 2, 4 GB memory. The system is realized with Microsoft Visual C++ 6.0 development environment based on the Intel® Open Source Computer Vision Library (OpenCV) [18]. Fig. 4 and Fig. 6 show two examples of image based non-photorealistic rendering, Fig. 5 and Fig. 7 are their close-ups respectively. Fig. 4(a) is the input image. Its stylized oil painting result is shown in Fig. 4 (b), which presents strong sense of layering as well as aesthetic feeling of skin texture, and embodies the traits of oil paintings. Fig. 4 (c) is the watercolor effect result with rich and colorful textures, in which the brighter tone strengthen it with more artistic feelings. The cartoon effect result is shown in Fig. 4(d), which preserves the image contours and equalizes the color distribution. The experimental results prove that the designed special visual effect filter for oil, watercolor and cartoon paintings can get satisfactory results with a few simple parameters, and is easy to use. 6. Conclusions This paper proposes an image based nonphotorealistic rendering method with image filtering for the non-professionals to get pictures into stylized oil, watercolor and cartoon painting effects. It effectively solves the existed problems in previous methods, such as complex models and complicated operations that require skilled user participation. We first summarize and analyze the characteristics and difficult technical points for the drawing modes of the oil, watercolor and cartoon paintings. Then we design appropriate special visual effect filters for them with the objective of realistic effect and easy to operate. Lots of experiments prove that our system is easy to use and can generate satisfactory artistic rendering results. Acknowledgements This work is supported by the National Natural Science Foundation of China under Grant No. 61003188, the open funding project of State Key Lab of Virtual Reality Technology and Systems at Beihang University under Grant No. BUAA-VR13KF-2013, the Zhejiang Provincial Key Laboratory of Electronic Commerce and Logistics Information Technology under Grant No. 2011E10005, the Education Department of Zhejiang Province under Grant No. Y201018011. References [1] . Jan Eric Kyprianidis, John Collomosse, Tinghuai Wang, Tobias Isenberg, State of the 'Art': A taxonomy of artistic stylization techniques for images and video, IEEE Transactions on Visualization and Computer Graphics, Vol. 19, Issue 5, 2013, pp. 866-885. [2] . Pravin Bhat, C. Lawrence Zitnick, Michael Cohen, Brian Curless, GradientShop: A gradient-domain optimization framework for image and video filtering, ACM Transactions on Graphics, Vol. 29, Issue 2, Article 10,2010, pp. 1-14. [3] . Cassidy J. Curtis, Sean E. Anderson, Joshua E. Seims, Kurt W. Fleischer, David H. Salesin, Computergenerated watercolor, in Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH' 97), Los Angeles, CA, USA, 1997, pp. 421-430. [4] . Aaron Hertzmann, Painterly rendering with curved brush strokes of multiple sizes, in Proceedings of the 25lh Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH' 98), Orlando, FL, USA, 1998, pp. 453-460. [5] . Hiroyasu Koshimizu, Masafumi Tominaga, Takayuki Fujiwara, Kazuhito Murakami, On KANSEI facial image processing for computerized facial caricaturing system PICASSO, in Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (IEEE SMC'99), Tokyo, Japan, Vol. 6, 1999, pp. 294-299. [6] . Xiaoyang Mao, Yoshiyasu Nagasaka, Atsumi Imamiya, Automatic generation of pencil drawing from 2D images using line integral convolution, in Proceedings of the 7th International Conference on Computer Aided Design and Computer Graphics (CAD/GRAPHICS' 01), 2001, pp. 240-248. [7] . Aaron Hertzmann, Paint by relaxation, in Proceedings of the Computer Graphics International (CGI'2001), Hong Kong, 2001, pp. 47-54. [8] . Aaron Hertzmann, Fast paint texture, in Proceedings of the 2nd International Symposium on NonPhotorealistic Animation and Rendering (NPAR 2002), Annecy, France, 2002, pp. 91-96. [9] . James Hays, Irfan Essa, Image and video based painterly animation, in Proceedings of the 3rd International Symposium on Non-Photorealistic Animation and Rendering (NPAR 2004), Annecy, France, 2004, pp. 113-120. [10] . David Small, Simulating watercolor by modeling diffusion, pigment, and paper fibers, in Proceedings of SPIE Conference on Image Handling and Reproduction Systems Integration, Vol. 1460, 1991, pp. 140-146. [11] . Thomas Luff, Oliver Deussen, Real-time watercolor for animation, Journal of Computer Science and Technology, Vol. 21, Issue 2,2006, pp. 159-165. [12] . Philippe Decaudin, Cartoon-Looking Rendering of 3D-Scenes, Research Report, INRIA No. 2919, Université de Technologie de Compiègne, France, June 1996. [13] . Johan Claes, Fabian Di Fiore, Gert Vansichem, Frank Van Reeth, Fast 3D cartoon rendering with improved quality by exploiting graphics hardware, in Proceedings of the Image and Vision Computing, (IVCNZ 2001), Dunedin, New Zealand, 2001, pp. 13-18. [14] , Jeff Lander, Shades of Disney: Opaquing a 3D world, Game Developer Magazine, Vol. 7, Issue 3, 2000, pp. 15-20. [15] . Siddharth Hegde, Christos Gatzidis, Feng Tian, Painterly rendering techniques: a state-of-the-art review of current approaches, Computer Animation and Virtual Worlds, Vol. 24, Issue 1,2013, pp. 43-64. [16] . Adrien Bousseau, Fabrice Neyret, Joëlle Thollot, David Salesin, Video watercolorization using bidirectional texture advection, ACM Transactions on Graphics, Vol. 26, Issue 3, Article 104, 2007, pp. 1-7. [17] . Yotam Livny, Michael Press, Jihad El-Sana, Interactive GPU-based adaptive cartoon-style rendering, The Visual Computer, Vol. 24, Issue 4, 2008, pp. 239-247. [18] . Intel® Open Source Computer Vision Library web portal (http://opencvlibrary.sourceforge.net). Zhenyang ZHU, Jiadi TANG, Chunxiao LIU and Weijie HE School of Computer Science & Information Engineering, Zhejiang Gongshang University, Hangzhou 310018, P.R. China E-mail: [email protected] Received: 10 October 2013 /Accepted: 22 November 2013 /Published: 30 December 2013 (c) 2013 International Frequency Sensor Association |