TMCnet News
Human Motion Capture System and its Sensor Analysis [Sensors & Transducers (Canada)](Sensors & Transducers (Canada) Via Acquire Media NewsEdge) Abstract: The approach taken in the paper is to compare the features and limitations of motion trackers in common use. Results from the author's experimentation with an inertial motion capture system are discussed. The system mainly involves inertia sensing technology, Bluetooth, sensor network and software development of human body motion capture model. The sensor network of the system is used to collect motion data of the body key joints, and the data are delivered to workstation through Bluetooth, then the software on workstation uses analytical inverse kinematics algorithm to analyze the motion data. The system has advantages of lower cost and high precision. The resulting model tends to handle uncertainty well and is suitable for incrementally updating models. There is value in regularly surveying the research areas considered in this paper due to the rapid progress in sensors and especially data modeling. Copyright © 2014 IFSA Publishing, S. L. Keywords: Sensors, Motion capture, Sensor network, Data acquisition, Bluetooth transceiver. 1. Introduction Motion tracking is a vital component of developing intelligent autonomous robots or human motion capture. A robot agent or human motion capture system must be able to perceive human motion in order to interact, co-operate, or imitate in an intelligent manner. In recent years, several different sensing mechanisms can track sufficient motion leading to wide variety of uses. Robotic and automation applications include surveillance and human interaction, which requires a form of identity or action recognition [1], teleoperation [2], robot programming by demonstration [3] or humanoid imitation [4], Essentially, motion is recorded by tracking the precise position and orientation of points of interest at high frequency. Each tracker uses fundamentally different physical principles to measure position and orientation. Reference [5] gave a tutorial on sensing technology range and focused on augmented reality applications. The most comprehensive is reference [6] which is primarily focused on advances in image processing for markerless motion capture rather than considering wearable marker motion tracking. However, significant motion capture advances were also summarized including a number of analyses of motion capture data. With the development and integration of computer technology, and in the virtual reality technology and animation game industry, the motion capture technology is widely used and has achieved greater and greater market. The interpenetration and fusion of art and technology is becoming a development trend in the future. The application of motion capture technology can solve the bottleneck problem of defining the motion trajectory of virtual role in the 3D animation, and decrease the workload of controlling virtual roles, and consequently increase the efficiency of game development. Due to the application of the increasingly widespread, some solutions of motion capture technology arise accordingly. The mainstream motion capture systems are the optical and mechanical ones [7]. The optical system needs too many high-speed cameras, so the price is higher and the workload of post-data processing is larger. The mechanical one has no specific requirement to the space, but good real time and high capture precision. The system can be used not only in 3D animation, but also in virtual reality, virtual training and simulation, more generally, nearly all kinds of motion measurement of animals or other objects. The purpose of this paper is to collate a compendium of recent approaches to human motion tracking in the context of robotic research in order to highlight potential advantages of each sensing mechanism, and shows an example of applications of mechanical human body motion capture technology, mainly involving inertia sensing technology, Bluetooth, sensor network and software development of human body motion capture model. 2. Tracking Technologies Available tracking techniques have been categorized based upon the working principle. Each motion tracking system has advantages that are useful depending on the application. 2.1. Optical-Passive Marker Optical detection with passive markers, or reflective indicators, uses multiple fixed high-speed cameras around the measurement area to triangulate a precise marker position. Infrared lighting allows the capture of high-contrast images of the reflective markers up to 2 kHz. At least two cameras at a time must capture a marker otherwise there are occlusion errors. Although markers cannot be differentiated from each other until post-processing analysis restores the correct path. This results set of unlabeled points in a 3D workspace that correspond to the kinematic structure of the subject. These optical systems are affected by instances of occluded markers but successful recordings have submillimeter errors. Redundant markers are often used to overcome occlusions which reduces the probably of error but increasing the number of markers also increases the processing latency. An advantage for passive marker systems is that the subject is not weighed down with battery packs or constrained by wires to sensors. Some significant disadvantages include portability and the measurement workspace, which is a small-fixed area in view of the cameras. The area can be increased but this is still limited by the space in an indoor venue and strength of the reflected light. 2.2. Optical-Active Marker Active optical markers act as a light source instead of a reflector and are often deployed as infrared emitting diodes. The light emission from markers is multiplexed and therefore the frequency of the camera speed is divided by the numbers of sensors to detect. Although this introduces a limitation on measurement frequency, less postprocessing required since individual each light emitting diode can be identified. Once again, the capture is limited by the arrangement of cameras and the field of view. The measurement area is typically in the order of several square meters, and theoretically higher than for passive systems because of the light intensity diminishing with the inverse square of the distance. Since the indicators are powered, for wireless recording the subject must wear power packs and secure wires that would otherwise impede motion. 2.3. Optical-Markerless Ideally, motion capture would only use one set of camera(s) from one angle, similar to human vision, without requiring any body markers. Although these vision-based processing techniques are a topic of research, the only accurate systems are confined to a restricted area and background, generally provide inaccurate estimates or require cameras from multiple viewing angles. Despite accuracy limitations 3D tracking from monocular images can recover gross motions [8]. Owing to the extensive research in vision-based processing, a more in-depth survey targeting this research can be found in [6]. Markerless motion capture is an ongoing research area with massive potential. It relies upon image segmentation and processing techniques to find a human posture which may be matched to a human template [9]. Common approaches employ background scene subtraction techniques to extract a silhouette [3] and various manifold learning algorithms [10]. 2.4. Inertial Inertial motion capture relies on acceleration and rotational velocity measurements from triaxial accelerometers and gyroscopes. Each inertial sensor positioned at strategic points on the body measures precise orientation to within 2° root mean square [11]. This is achieved with estimation techniques such as Kalman filtering [12] fusing the angular rate with incline (gravity vector) and, for some sensors, magnetometers for more reliable heading data. Assuming certain configuration for the sensors and calibrating the actor dimensions an accurate posture can be resolved. An advantage this system has over optical methods is the flexibility of recording in many environments. A major drawback of these sensors is estimating the position by integrating accelerations or angular velocity, a cumulative error arises, referred to as drift. Modem inertial motion capture suits rely upon ground contact force detection, indicated by sudden foot accelerations, to update reference position. Without well-defined events such as these the posture remains accurate but tracking world position is unreliable due to drift. Other limitations include the need for post-processing in uncertain environments, when the ground support is varying dramatically. Despite inherent problems associated with this technique, it is improved in combination with other technology. Reference [13] used inertial sensors with ultrasonic detection for a practical outdoor capture technique. With one optical marker, the suit may be tracked accurately within the camera workspace. 2.5. Magnetic Electromagnetic fields are established through precise current pulses in mounted transmitting antennae. Each magnetic field including the earth magnetic field is measured giving an estimation of joint position, angles and global orientation. AC electromagnetic systems are highly distorted by neighbouring metallic objects but recent DC magnetic field systems exhibit significantly reduced distortion. A tri-axial transmitter produces DC pulses sequentially to each axis and the receiving antennae, mounted on significant positions on the body, measure the magnetic field along each axis. The earth magnetic field is measured when no pulse is present and subtracted when measuring the orientation. This results in six degrees of freedom (DOF) position and orientation information for each sensor up to a range of 10 ft from the transmitter. A range of sensor configurations has been employed in robotics and action recognition tasks [14]. Advantages of this approach include the flexibility in locating the sensors on the body, there are no occlusion issues. The measurement area is limited to a small region around the transmitter, comparable to optical systems, and is as portable as the transmitter. Metallic objects still cause a significant level noise and distortion to measurements. 2.6. Mechanical The simplest method of capturing pose is to measure orientation directly using electromechanical potentiometers measuring the orientation displacement of each joint. This approach is effective in many cases since it is not affected by external forces or occlusions, measurements can be fast and the equipment portable. The main disadvantage is that motion is usually constrained by the rigidity of the wearable equipment. An exo-skeletal frame normally imposes restrictions on the range of motion since human joints are more flexible than the mechanical links. Another problem is in detecting the true position and orientation of the entire frame. This mechanism cannot detect events such as jumping or turning, only the relative angle between limbs. Therefore, captured results appear to slide, a problem that can be overcome by incorporating other measurement techniques. This method is particularly strong in exoskeletal frames and prosthetics since the joints must also be powered. 2.7. Acoustic By attaching ultrasonic transmitters and microphones at specific locations on a moving body, an estimate of position can be determined through the intensity of acoustic pulses. The pulses are multiplexed so that each microphone measures the pulse intensity from each transmitter to estimate the relative distances between all sensor points. A complication arising from this arrangement is self-occlusion, that is, parts of the moving body blocking a direct path to receiving microphones. It is especially difficult with partial occlusions since the reduced intensity should not be related to distance. Depending on the frequencies used the system is susceptible to background noises, temperature and humidity in uncertain environments, and to wind when used outdoors. 3. Application The application system discussed in this paper divides human body into 17 key joint points, and inertial sensors are set on the corresponding key points. All of the 17 sensors constitute a sensor network. The sensor network is used to obtain the real-time motion data of key joint points of the human body. Each inertial sensor consists of three internal sensors such as tri-axial acceleration sensor, tri-axial angular velocity sensor and tri-axis magnetic sensor, and MCU, as shown in Fig. 1. The inertial sensor used in this system applies Kalman filter algorithm to integrate and analyze the raw measurement data of internal three sensors. The inertial sensor outputs the motion data in the form of Euler angle or Quaternion, and the data are translated from the inertial sensors to the reconstruction software of the human body motion model on the workstation through Bluetooth, and the software analyzes and calculates the relative displacement of key joints through analytic inverse kinematics algorithm [15], and displaying the real-time human body motion posture graphically. At the same time the software can output storage file with the suffix .BVH (Biovision Hierarchy), thus the human body motion capture system realizes the real-time reconstruction of human body motion model. The principle of human body motion capture system is shown in Fig. 2. According to the way of processing motion data the human body motion capture system can be divided into two modules, the data acquisition module and the data processing module. Data acquisition module mainly collects the rotation angle data of 17 key joints through sensors network, and it is the basis of the real-time reconstruction of human body motion model. The data processing module is used to process the measurement data of the inertial sensor of each joint point, and retrieve the performer's motion posture data, and then the system software graphically displays the real-time motion posture of the virtual role corresponding to the true performer. 3.1. Data Acquisition Module 3.1.1. Design of Sensor Network Node The sensor network is mainly used to obtain motion data of the key joints of human body. In this paper, the system abstracts the human body motion into the movement of the human skeleton model, and human skeleton is a joint chain structure which is composed by a series of the sequentially connected rigid bodies, and here the junction of two rigid bodies is so called joint. So the human skeleton is divided into 17 critical joints, and the whole-body motion posture is built through integration and calculation of the motion data of 17 joints. Therefore, the motion of the whole body can be viewed as the motion of human skeleton model which is connected by 17 critical joints, and hip abdominal joint is taken as the root of joints to determine world space position and orientation of human body. Here stratified skeleton structure (i.e., tree structure) is used for graphic description, as shown in Fig. 3. The process to build hierarchical model of human body joints is a procedure to extract key joint points of human body because the motion data of key joint points can reflect the motion posture of the whole human body after the motion data are processed. In order to obtain the motion data of 17 key joint points, the system places 17 inertial sensors in the corresponding key points, so the sensor network of the application contains 17 sensor nodes and each sensor gets motion data of the specific joint point. In the reconstruction software of human body motion model, the motion data of 17 key joint points are analyzed and calculated through analytical inverse kinematics algorithm, then the relative displacement of each joint point is obtained by the software, and the true performer's motion posture is reconstructed at last. Therefore, the distribution of sensor network nodes is determined by the hierarchical model of human body joint, as shown in Fig. 4, and each black point represents a sensor network node in the diagram. Inertial sensor used in the sensor network captures the real-time motion data of the key joints, then integrates and calculates the measurement data of internal three sensors through the Kalman filter algorithm [16]. Another feature of the inertial sensor is the internal integration of the three types of sensor units, and the measurement data of the three kinds of sensors compensate each other, so the inertial sensor can measure the optimum motion data. The principle and function of inertial sensors in the system is shown in Fig. 5. Inertial sensor also has the parameter adjustment function. If a ferromagnetic substance exists in the working environment or field magnetic intensity of working environment is unstable, user can try to reduce the sensitivity of the magnetic sensor, and relatively increase the sensitivity of angular velocity sensor and acceleration sensor to adapt to the environment changing. The tights sensor fixtures arranged as the sensor network is shown in Fig. 6. 3.1.2. Design of Master Controller System There are two master controllers in the motion capture system, which respectively obtain the right and left bust motion data from sensor network nodes. The master controller utilizes the ARM chip STM32F103RE. The chip well meets the requirements of real-time data acquisition and transmission of the system, and of constant data collections of each network node. Each network node connects to master controller with RS-485 bus of a higher data transfer rate (10 Mbps). The interface of the bus also has advantages of good anti-noise and longer transmission distance. The Bluetooth transmission module is integrated in the master controller system, which is used to wirelessly send motion data of key joints to the workstation (PC). So the master controller system can transmit the realtime motion data to the reconstruction software of human body motion model on the workstation. Fig. 7 shows the physical master controller hardware module. 3.1.3. Design of Bluetooth Transceiver Bluetooth technology is an open global specification of wireless data and voice communication, which based on low-cost and lowpower wireless connection [17]. Bluetooth transmission module of the system discussed in this paper uses Bluetooth chip AUDIO-FLASH made by the CRS company, and adds Atmel T7024 Bluetooth chip which dedicated to front-end chip to be expand into standard class 1 Bluetooth module. It is one of innovative technologies of the human body motion capture system to apply Bluetooth for realization of wireless transmission of the motion data. The wireless data transmission between master controllers and the workstation is done by sender and receiver of Bluetooth transmission modules integrated respectively both in the master controller and workstation, as shown in Fig. 5. 3.2. Data Processing The motion data processing is completed by the software of motion capture system. The reconstruction software of human body motion posture should have the functions of intelligent data processing and real-time reconstruction of the motion posture, and at the same time it can also edit and modify the recorded file. The development of the software is based on the integrated development environment of Visual C++ 6.0, completed with C++ programming language and the techniques of MS Comm communication controls, MFC, COM components, and OpenGL. The software of human body motion capture involves two parts, the motion posture display module and system configuration module. The system configuration module communicates with Bluetooth devices of specified ports, and obtains the motion data of 17 key points of the whole body. Communication between the reconstruction software of motion model and Bluetooth devices is realized through MSComm controls and the serial ports. MSComm control provides the OnComm events and CommEvent attribute to capture and monitor the values of communication events, and functions of Input and Output can be used to read or write character data to the corresponding buffers. The motion posture data collected from Bluetooth module is delivered to the motion posture display module of the software, which is mainly used to display performer's real-time posture in three-dimensional scene, and then the synchronization of motion posture of the virtual role and true performer is realized. The accuracy of motion capture depends on the algorithm to calculate the human body motion data, namely it depends on that how to use the data of each sensor for getting the performer's motion posture. The system analyzes and calculates the motion data through analytical inverse kinematic algorithm, which can effectively improve the calculation speed and real-time characteristics of the system [18]. Inverse kinematics algorithm uses the formulae directly to calculate the motion posture data of middle joints of the joint chain through the known location and status of the end effector, which consequently improve the efficiency of motion capture. Fig. 8 shows the motion posture that the performer is waving left hand. 4. Conclusions The various advantages and disadvantages of each sensor mechanism are discussed and the development process of mechanical human body motion capture system based on inertial sensors is described in the paper. The expansion in motion capture use may be driven by the portability and relative cost effectiveness of non-optical technologies. Optical systems, while achieving higher measurement accuracy, are often limited to fixed laboratory spaces. The portability of an inertial system leads to automation applications that are less constrained by the environment and could lead to greater deployment in industry. The system as an example of application in this paper has three features. Firstly, it collects the motion data for the key joint points of human body through inertial sensor network. Secondly, the system uses the Bluetooth technology to realize the wireless transmission of the human body motion data, which effectively improves the flexibility of the actions of the performers. Thirdly, since each of sensors of the system has a higher relative independence, it is convenient to expand the study of the motion capture system, such as the research of multiplayer real-time motion capture technology, which will be the next step to do the further research. References [1] . J. A. Ward, P. Lukowicz, G. Tröster, and T. Stamer, Activity recognition of assembly tasks using bodyworn microphones and accelerometers, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 28, Issue 10,2006, pp. 1553-1567. [2] . N. Miller, O. C. Jenkins, M. Kallmann, and M. J. Mataric, Motion capture from inertial sensing for untethered humanoid teleoperation, in Proceedings of the 4th IEEE/RAS International Conference on Humanoid Robots, Los Angeles, CA, 10-12 November, Vol. 2,2004, pp. 547-565. [3] . P. Azad, T. Asfour, and R. Dillmann, Toward a unified representation for imitation of human motion on humanoids, in Proceedings of the IEEE International Conference on Robotics and Automation, Roma, 10-14 April, 2007, pp. 2558-2563. [4] . S. Nakaoka, A. Nakazawa, K. Yokoi, and K. Ikeuchi, Leg motion primitives for a dancing humanoid robot, in Proceedings of the IEEE International Conference on Robotics and Automation (ICRA'04), New Orleans, LA, USA, Vol. 1, 26 April - 1 May 2004, pp.610-615. [5] . G. Welch, and E. Foxlin, Motion tracking: no silver bullet, but a respectable arsenal, Computer Graphics and Applications, Vol. 22, No. 6, 2002, pp. 24-38. [6] . T. B. Moeslund, A. Hilton, and V. Kruger, A survey of advances in vision-based human motion capture and analysis, Computer Vision and Image Understanding, Vol. 104, No. 2,2006, pp. 90-126. [7] . Huang Boshi, Chen Fumin, Research on human body motion capture and motion control, Computer Engineering and Applications, Vol. 41, No.7, 2005, pp.60-63. [8] . A. Agarwal, and B. Triggs, Recovering 3D human pose from monocular images, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 28, Issue 1,2006, pp. 44-58. [9] . L. Sigal, A. Balan, and M. Black, HumanEva: synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion, International Journal of Computer Vision, Vol. 87, No. 1-2, 2009, pp. 4-27. [10] . A. Elgammal, and C.-S. Lee, Tracking people on a torus, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 31, Issue 3, 2009, pp. 520-538. [11] . A. G. Cutti, A. Giovanardi, L. Rocchi, and A. Davalli, A simple test to assess the static and dynamic accuracy of an inertial sensors system for human movement analysis, in Proceedings of the 28,h IEEE EMBS Annual International Conference, New York, NY, USA, August 30-September 3, 2006, pp. 1002-1009. [12] . D. Roetenberg, H. J. Luinge, C. T. M. Baten, and P. H. Veitink, Compensation of magnetic disturbances improves inertial and magnetic sensing of human body segment orientation, IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 13, Issue 3,2005, pp. 395-405. [13] . D. Vlasic, R. Adelsberger, G. Vannucci, J. Barnwell, M. Gross, W. Matusik, and J. Popovic, Practical motion capture in everyday surroundings, ACM Transactions on Graphics, Vol. 26, Issue 3, 2007, pp. 35:1-35:9. [14] . T. Yamamoto, and T. Fujinami, Hierarchical organization of the coordinative structure of the skill of clay kneading, Human Movement Science, Vol. 27, No. 5,2008, pp. 812-822. [15] . Anders Lau Olsen, Henrik Gordon Petersen, Inverse kinematics by numerical and analytical cyclic coordinate descent, Robotica, Vol. 29, No. 4, 2011, pp. 619-626. [16] . Cao Chun-Ping, Luo Ling-Li, Wireless indoor positioning system based on Kalman filtering algorithm, Computer Systems & Applications, Vol. 20, No. 11,2011, pp. 76-79. [17] . You Ke, Liu Ruiqiang, Bluetooth-based wireless sensor networks, in Proceedings of the 2nd International Conference on Space Information Technology, Wuhan, China, 2007. [18] . Wang Yi-Gang, Wu Zi-Zhao, Chen Xiao-Diao, Kind of analytical inverse kinematics method, Application Research of Computers, Vol. 26, No. 06, 2009, pp. 2368-2369. Guangtian Shi, Yongsheng Wang, Shuai Li Lanzhou Jiaotong University, Lanzhou, China Tel: 086-931-4938013 E-mail: [email protected] Received: 14 April 2014 /Accepted: 30 May 2014 /Published: 30 June 2014 (c) 2014 IFSA Publishing, S.L. |