TMCnet News
Analyzing and Improving the Camera Plane Calibration Method [Sensors & Transducers (Canada)](Sensors & Transducers (Canada) Via Acquire Media NewsEdge) Abstract: An improved method for camera plane calibration is proposed by analyzing and testing the two-step calibration method based on plane. In the improved method, central area of images is used firstly to solve initial values of the camera intrinsic parameters, improving precision of the initial values. Then, to facilitate dimension measurement using machine vision, the lens distortion model is modified and the projection direction is changed. Experiment results show that the improved calibration method could increase camera calibration precision, and easy to operate. It can be used in the machine vision research and the industry 3D measurement. Copyright © 2013 IFSA. Keywords: Camera calibration, Distortion model, Intrinsic parameters initial value, Residual error, Comer point field. (ProQuest: ... denotes formulae omitted.) 1. Introduction In the field of machine vision research and the industry 3D measurement, camera calibration plays an important role. People require continuous improvement of the calibration precision and its convenience. Camera calibration technology mainly includes the classical calibration method and the self calibration method. Classical calibration method uses a known reference object to solve the camera parameters by building imaging model, adopting the method of mathematical transformation and optimization. It can divide into linear method and nonlinear method, and two-stage method. The Camera linear calibration method ignore lens distortion and simple calculation, But the accuracy of the calibration are generally not precision. Nonlinear calibration method considering the camera lens distortion, accuracy in calibration is better than linear method, but solving more complex [1]. R. Y. Tsai method [2], J. Heikkila method [3] and Zheng-you Zhang method [4] are Classic camera calibrations. At present of classic calibration technology research, it is mainly focus on nonlinear distortion correction. Self calibration method [5, 6] mainly includes camera self-calibration technology of active vision, selfcalibration technology of essential matrix and basic matrix, etc. self calibration method don't need reference objects, direct calculate the camera parameters only use the relationship among the corresponding points on different images. Self calibration method is quite flexible, but the process is pretty complicated, and besides it lack of robustness, because it must solve the multielement nonlinear equation. It mainly applies in the fields of low precision requirement. We often use the classical calibration method in dimension measurement; the most common is Tsai method, Zhengyou Zhang's method, etc. R. Y. Tsai proposed that the two-stage calibration method, it is based on radial alignment constraint (RAC). The first is established the linear equations of calibration initial value, and to resolve the part of parameters, and then using the nonlinear optimization solution to complete rest of the parameters. Owning to the most of equation is linear equation, and the operation is simple. But the method is only considering the radial distortion, and also need the three-dimension target of high precision, besides calibration is high in cost. J. Heikkila's method at first is ignoring distortion. System of linear equations is built according to the transformation relationship, which the space point of camera imaging projected to the corresponded pixel, then considering the optimized solution of radial direction and tangential distortion. Zhengyou Zhang's two-stage calibration is considering the first two items of the lens radial distortion. At the first, this method is uses the linear transformation for solving camera parameters, as the initial value of nonlinear optimization and then considering radial distortion, using optimization algorithm for solving nonlinear equations. As so far, this approach is being applied widely. However, calibration precision of two steps is related to the select of initial value of nonlinear optimization. If select of initial value will be proper, the convergence result will be good. But if the condition is in the contrast, the result will be not good. On the other side, the lens distortion model of Zhengyou Zhang's method is not convenient to using pixel information to measuring the size of the object. And the hypothesis of the radial distortion only is inconsistent with the actual conditions; that limit the further improvement on the calibration precision. This paper aims at the limitation of the plane two-stage calibration method, analyzes effect of initial value on the calibration results, then put forward an improved calibration method. Through the experiment to make sure the image area of calculation nonlinear optimization initial value, consider the Lens of radial and tangential distortion, and modify the projection direction of lens distortion model. Through the contrast experiment of the two calculation methods, the paper draw the conclusion that improved method can improve the camera's calibration accuracy and make use of dimension measure. 2. Camera Imaging Model The perspective projection relationship between 3D space point and 2D image point in the pinhole camera model is reflected by use of the tow groups of parameters. Imaging principle is as shown in Fig. 1. Four coordinate systems would built in pinhole camera model, can be defined as follows [7]. Pixel coordinate system (PCS) shows the pixel site of a pixel point in the image; Its origin point ot is located in lower left quarter of the image plane, the coordinate axis Xu and axis yu are parallel to the pixel column and the pixel row, whose unit is pixel. Image coordinate system (ICS) shows real physics site of the pixel point in the image. Its origin point o (principal point) is the intersecting point of the optical axis of the camera with the image plane, coordinate axis X and y are parallel to the axis Xu and axis yu, whose unit is real physics size. Camera Coordinate System (CCS), origin point oc is camera optical center, coordinate axis zc is optical axis, coordinate axis xc and yc are parallel to the axis X and axis y separately, and have the same direction. It shows the site of the pixel point in the 3D space whose central is camera. World coordinate system (WCS) is defined to show the camera's site in the shoot environment, origin point Ow and coordinate axis Xw,yw,zw can set up according to the specific application environment. 3. The Problems in the Camera Plane Calibration Method Based on the Zhengyou Zhang's two steps calibration, At the first, this method is uses the linear transformation for solving camera parameters as the initial value, and then considering the lens distortion, to resolve the minimization problem of nonlinear as improving the accuracy in calibration [8]. Specific calibrated model are as follows. 3.1. Camera Plane Calibration Method A 2D point in the image coordinate system is denoted by m . A 3D point in the world coordinate system is denoted by M . M and M is the homogeneous vector form of them. Camera linear imaging model is ...(1) where H is the homography matrix, H = K\rx r2 T], K is the intrinsic matrix, and it has five intrinsic parameters CC, ß, y, jUq , VQ . And [ji r2 T\ is the extrinsic parameters, it has six extrinsic parameters 0,if,i//,Tx,Ty,Tz. Using the method of linear to solve homography matrix Hi initial value, then using the method of nonlinear optimization to solve Hi , and the optimization object function is: ...(2) The distortion model is ...(3) where kx,k2 are the coefficients of the radial distortion, r2 = x'2 + ÿ2. Optimized all parameters by function ...(4) kx,k2 are the coefficients of radial distortion. px, p2 are the coefficients of tangential distortion. The coordinate transformation procedure is as shown in Fig. 2. In Fig. 2, JU0,VQ -Image main point coordinates, f-Scale factor. 3.2. The Influence of Intrinsic Parameters Initial Value to Camera Calibration This Zhengyou Zhang's two steps calibration method is using the comer of whole image to solve the intrinsic parameters initial value. However, considering the process of solving linear initial value, if taking the scope of central image is too big or small, it will makes contained comer point in the field of area limited and bring a little bit distortion, leading to the initial value not accurately and makes the final calibration error. Therefore, adopted the proper image central is important for improving initial value to solving precision and calibration accuracy. According to the calibration procedures, make sure that surrounding the image of central at calibration board, selected in the field of 180^180, 300x300 and 600x600 pixels comer point coordinates, respectively, to calculate as the initial value of intrinsic parameter. As shown in Fig. 3. Then we use the calculated intrinsic parameter initial value to ensure the calibration parameters, calibration results and residual of nonlinear optimization which are determined by least squares, as Table 1. As is shown at Table 1, according to the size of the region is different of calculation initial value selection will leading to get the different initial value. However, the different of the intrinsic parameter initial value will cause the different residual error of least squares optimization. Therefore it will affect the final calibration results. This is a problem in the camera plane calibration method. 3.3. The Distortion Model of Plane Calibration Method It is only considered the first two terms of radial distortion in the distortion model of Zhang's method, see the formula (3). Dimension measurement, is that, based on detection of pixel coordinates m of image to get the object world coordinate M . It can convenient for actual measurement. To the two steps plan calibration, it is need that distortion model (3) type to solving equation of higher degree, besides the process of calculation is too complex to get the analytical solution. The only way is through the method of optimal iterative method to solve the numerical solution. This solution need amounts of calculation and low efficiency, but also it affected that accuracy limited of the intrinsic parameter initial value, ultimately the optimization solution will exists a lot of errors. This is another problem in the camera plane calibration method. 4. Improving the Plane Calibration Method To the two questions that mentioned in the third section, we adopt the improving method as follows. 4.1. Ensure the Image Area of Intrinsic Initial Values Calculation By the analysis of experiments, Camera calibration precision is relevant with the image area which selected to calculate the intrinsic parameter initial value. From (3), comer point far from image centre makes great difference to the lens distortion. If the distortion of the comer point is too small to ignore, and the quantity is big enough, its Pixel coordinates m is approximate match the linear imaging model with the world coordinate M that makes the solution of the intrinsic parameter initial value more accurately. Next, ensure the appropriate region of calculated intrinsic parameter initial value by experiment [9]. The resolution of the camera that collect the images of the calibration boards is 1392x1040 pixels, CCD photosensitive unit size is 6.45x6.45 pm, the focal length of the lens is 25 mm, calibration board is 75x75x3.0 mm, shooting 20 different poses of the calibration board image to calibrate the camera parameters. Specific methods are as follows: Around the center of the image, start with 150x150 pixels, each increase 5 pixels to expand the area to select a comer, until 1020x1020 pixels, solving the camera's intrinsic parameter initial value, as shown in Fig. 4(a), Fig. 4(b) indicate the comer area selected is 300x300 pixels, but the different poses four images of calibration board posture. Using the Bouguet algorithm [10] to detect the pixel coordinates of the selected comer point that inside the Fig. 4, then accordance with the calibration relationship of Zhang, separately calculate the intrinsic parameter initial value of the imaging model, and then using modified distortion model (6) get the calibration results of each cases and the nonlinear optimization residuals. For 20 images, select different size of comer area respectively, conducting the camera calibration, the intrinsic parameter initial value, calibration results and the optimization of residuals, which calculated 150^150, 300x300, 500x500, 800x800, 1000x1000 pixels is as shown in Table 2. From Fig. 5, the optimization residual of calibration depend on the area of selected comer point. When the area is less than 250x250 pixels, due to contained comer point less as results to final calibration optimization may have bigger residual error. When the area is between 250x250 pixels and 500x500 pixels, optimization residual of calibration nay have smaller, it is about 0.005 mm. In the case, solving the size of area is proper and the calibration precision is higher. But when the size of area is more than 500x500 pixel, because containing the comer point with large distortion leads to residual error increase gradually so as to lowing the calibration precision. Therefore, when the image center area is between 250x250 pixels and 500x500 pixels, it will be better. This paper will adopted 300x300 pixels in the later experiments. 4.2. Modify Distortion Model At first, this paper fixed the projection direction of distortion model can avoid to solving equation of higher degree and makes calculation process is simple when solve the ideal image Coordinate (x',y')T based on the real image coordinate (x,ÿ)T , finally improving the stability and accuracy of measurements. Modified projection direction, from (3), we have ... (5) Secondly, it is only hypothesized that camera lens have radial distortion, which satisfy (3), but do not consider the tangential distortion. It is deviate from the actual situation, and limits the further improvement of calibration precision. In order to improve the accuracy of measurement, based on modified the projection direction of distortion model, this paper adopt J. Heikkila's distortion model, introducing the first two terms [11] of tangential distortion. The specific method of modify distortion model as follows: ... (6) 5. Improved Camera Plane Calibration Experiments 5.1. Improved Calibration Algorithm Based on the analysis, in the formula (1) of camera linear calibration use the image center area of 300^300 pixels to solving intrinsic parameter initial value; In the process of camera nonlinear calibration, adopted lens distortion type (6), which is the way which used to improved camera calibration method. The modified calibration process can be seen in Fig. 6. The first, based on the camera linear imaging model, we solve intrinsic parameters and extrinsic parameters, then optimize all the parameters(see column 1 in Fig. 6); secondly, considering the lens distortion, based on the nonlinear imaging model to solve the initial value of distortion coefficient, then optimize them with intrinsic parameters and extrinsic parameters(see column 2 in Fig. 6); the third, solve the extrinsic parameters of the image 1 (see column 3 in Fig. 6). The method can calibrate respectively the intrinsic parameters of camera and extrinsic parameters required in actual measurement to be used in practice. 5.2. Experiments Analysis Considering in the experiment, this paper adopted the camera's photosensitive unit is a square. Therefore, suppose CC= ß, y = 0 in the intrinsic parameter. Take image at centre point area for 300x300 pixels, adopted improved calibration method and the original plane calibration are calibrated respectively. Again, Shooting two groups images of the calibration board for calibrated, respectively. The calibration result of 3 groups as shown in Table 3. It is thus clear from Table 3, the least squares optimized residual error is smaller by improved calibration method than plane calibration method, so that the camera calibration precision is higher. 5.3. Influencing Factory Analysis of Calibration The process of camera imaging is very complicated. Lots of factors can affect the imaged results, such as light, temperature, dithering, etc, that universally exist objectively. The main factors that affect the results after the repeated test as follows: 1. Extracting comer point and measurement calibration board There are i (i = 1,2,3,...) comer points in the image that satisfy (1). It is obviously that extracting comer points and measurement calibration board can affect the solution of the homography matrix. And they can cause error on calibration. 2. The number of calibration points When solve the simultaneous equations to obtain the intrinsic parameters and extrinsic parameters, we only need 12 equations, that is, we only need 6 coordinates of calibration points. But it can affect the results, some results close to the fact and some differ a lot from the fact, calibration results are not quite robust. In calibration, we do our best to solve accurately each point in the image. To cause the calibration result become smooth and unify, we increase the number of calibration points in order to make the result more precise. In fact, calibration point number is far greater than 6 when we calibrate. 3. The Number of images We take an image at a different angle that required in the calibration method, that is, the image is a field of view. From (1), we can unique solve the homography matrix H . The number of images determines the number of H. So in the processing of solve the intrinsic parameters and extrinsic parameters by homography matrix, the number of H can influence the calibration precision. There are 9 camera parameters if don't consider the distortion, and 8 degree of freedom can be obtain by a homography matrix, so we need 2 images at a different angle at least in the calibration. A large number of images usually collected in actual application to obtain the stable result. That will take the error to the intrinsic parameters if the images collected not enough. By the experiment, we see that each parameter calibration value tends to become stable when the number of images greater than 8. But if images are collected too much, which will influence the calibration precision by the much extracting comer points. On the other hand, it can increase the error probabilities because the accumulative error that brought by too many images. The right number of images must be considered in calibration. We think it is proper to adopt 9 to 11 images. In experiment above, we all adopt 9 images. 6. Conclusion The method of improving camera plane calibration is put forward based on the analysis of plane calibration method. Firstly, ensure the image center area of solving nonlinear optimization initial value; it is suitable to pick from 250^250 pixels to 500x500 pixels of the comer point of Image center area to calculate the initial value; secondly, improve the camera lens distortion model, modifying the projection direction of distortion model, and introducing the tangential distortion. Experiments show that, the methods in this paper can reduce the residual error of nonlinear optimization, and enhance calibration accuracy of the camera. Furthermore it has higher speed and easy to operate by use 9 images of plane targets, can apply to all kinds of distortion models of lens. It has more applicability than Zhang's method that only considered the radial distortion, and makes use of dimension measure. In this paper, we don't research the factors produce the effect on the camera calibration, such as the aperture, the focal circle of a camera etc. These problems will need to be further study. Acknowledgements The work is supported by both science technology project of education department during Eleventh Five-Year Plan (2010384) and Industrial technology research and development project in the Jilin province (2012747). References [1]. Chi Jian-Nan, Vision Measurement Technology, China Machine Press, 2011. [2]. R. Y. Tsai, A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses, IEEE Journal of Robotics and Automation, Vol. 3, No. 4, 1987, pp. 323-344. [3]. J. Heikkila, Geometric camera calibration using circular control points, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, Issue 11, 2000, pp. 1066-1076. [4]. Zhengyou Zhang, A flexible new technique for camera calibration, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, No. 11, 2000, pp. 1330-1334. [5]. Cheng Lei, Zhan-Yi Hu, Fu-Chao Wu, et al., A novel camera self-calibration technique based on the Kruppa equations, Chinese Journal of Computers, Vol. 26, No. 5,2003, pp. 587-597. [6]. Meng Xiao-Qiao, Hu Zhan-Yi, Recent progress in camera self-calibration, Acta Automática Sínica, Vol. 29, No. 1,2003, pp. 110-124. [7]. Xu Jie, Analyzing and improving the Tsai camera calibration method in machine vision, Computer Engineering & Science, Vol. 32, No. 14, 2010, pp. 45-58. [8]. Jia Dan, Study on algorithm of on-the-job camera calibration, Harbin Engineering University, 2007. [9]. Wei Guang, Research on machine vision technology for measuring shaft diameters, Jilin University, 2011. [10]. Jean-Yves Bouguet, Pyramidal implementation of the Lucas Kanade feature tracker description of the algorithm, Technical Report, OpenCV Document, Intel Microprocessor Research Labs., 2000. [11]. Zheng Jun, Gong Sheng-Rong, Liu Chun-Ping, Nonlinear camera calibration method using corrected template, Computer Engineering and Applications, Vol. 47, No. 31,2011, pp.192-196. 2 Chun-Yan WANG,1 Qing-Chang TAN, 3 Guang WEI,1 Guan-Nan LI 1 College of Mechanical Science and Engineering, Jilin University, Changchun 130022, China 2 School of Computer Science and Technology, Changchun Normal Universityô Changchun 130032, China 3 China FAW Company Limited R&D Center, Changchun 130011, China 1 Tel.: 0431-86168123 E-mail: [email protected] Received: 19 August 2013 /Accepted: 25 October 2013 /Published: 30 November 2013 (c) 2013 International Frequency Sensor Association |