TMCnet News

Development of Human Motion Capture System Based on Inertial Sensors 2125 [Sensors & Transducers (Canada)]
[July 17, 2014]

Development of Human Motion Capture System Based on Inertial Sensors 2125 [Sensors & Transducers (Canada)]


(Sensors & Transducers (Canada) Via Acquire Media NewsEdge) Abstract: Motion capture systems play an important role in health-care, sport-training systems and cartoon animation. This paper develops a motion capture system with inertial motion trackers for three-dimensional animation production. The system mainly involves inertia sensing technology, Bluetooth, sensor network and software development of human body motion capture model. The sensor network is used to collect motion data of the body key joints, and the data are delivered to workstation through Bluetooth, then the software on workstation uses analytical inverse kinematics algorithm to analyze the motion data. Human body motion capture system mainly involves the design of human body model and development of real-time reconstruction software. A human body is abstracted into 17 key joints affecting human movement for model, and inertial sensors are put on each joint point to measure the motion data of the node, so a 17 nodes sensor network is built. The experiment shows that the model tends to handle uncertainty well and the software and hardware of motion capture system have advantages of good consistency of virtual character and performer athletic stance and farther effective communication distance, and it is very easy to further expand the research of this subject, such as the design of multiplayer real-time motion capture system. Copyright © 2014 IFSA Publishing, S. L.



Keywords: Inertial sensors, Motion capture, Sensor network, Inverse Kinematics, UWB positioning.

1. Introduction In recent years, more and more people are interested in outdoor sports and leisure activities [1], Consequently, the cartoon animation production and sport equipment industry have been dramatically grown up in recent years [2], Such phenomena are particularly evident in golf and tennis industries [3]. Not only do the young people want to enjoy their daily lives but also many elderly people are eager to prolong their active and independent lives by continuously monitoring their health conditions. Such great interests have resulted in a great demand on a motion capture system. The vision-based motion capture systems (MCSs) have enabled the amateur athletes to quantitatively monitor their postures and to improve the records by self-practice. For example, Chan et al. developed a virtual reality dance training system based on a vision-based MCS [4], Although these MCSs may help the amateur athletes learn the sport postures and improve the records, their effectiveness is significantly limited in practice, because the vision-based MCSs are usually available only in a special setting, i.e., the background and the light condition should be limited to obtain good motion capture. The drawbacks are not only the restriction of natural motions but also the restriction of the condition where the capturing is conducted.


Because of high cost of the vision-based MCSs most of MCSs are now in the form of a wearable sensor system, which does not constrain the space for motion capturing. For example, Kong and Tomizuka proposed Smart Shoes and a gait analysis algorithm that detects the likelihoods and abnormalities in human gait phases [5]. Hashi et al. developed a motion capture system with a customized magnetic core [6]. In medical applications, Kwon et al. developed a knee joint motion capture system based on the surface-electromyography and verified the developed device with human subjects with cerebral palsy [7].

None of these systems, however, provides the complete information on the whole body motion in an open three-dimensional space yet.

An inertial measurement unit (IMU) is a common option for the development of MCSs. The IMUs in general consist of three-axe accelerometers, three-axe gyroscopes, and three-axe magnetometers; an IMU measures the direction of gravity, motion (i.e., translation and rotation) accelerations, turning rates, and the direction of earth-magnetic field. By the fiision of this information, it is possible to estimate the orientation of a rigid body in the threedimensional space.

Therefore, it may be able to estimate the whole body motions, if the IMU is attached on everybody segment of a user. It should be noted that the accuracy of the attitude estimation by an IMU is highly dependent on the performance of the sensor fiision algorithm.

Inertial motion capture relies on acceleration and rotational velocity measurements from triaxial accelerometers and gyroscopes. Each inertial sensor positioned at strategic points on the body measures precise orientation to within 2° root mean square [8]. This is achieved with estimation techniques such as Kalman filtering [9] fusing the angular rate with incline (gravity vector) and, for some sensors, magnetometers for more reliable heading data. Assuming certain configuration for the sensors and calibrating the actor dimensions an accurate posture can be resolved. An advantage this system has over optical methods is the flexibility of recording in many environments.

A major drawback of these sensors is estimating the position by integrating accelerations or angular velocity, a cumulative error arises, referred to as drift. Modem inertial motion capture suits rely upon ground contact force detection, indicated by sudden foot accelerations, to update reference position. Without well-defined events such as these the posture remains accurate but tracking world position is unreliable due to drift. Other limitations include the need for post-processing in uncertain environments, when the ground support is varying dramatically.

Despite inherent problems associated with this technique, it is improved in combination with other technology. Vlasic et al. used inertial sensors with ultrasonic detection for a practical outdoor capture technique [10].

2. MCS based on IMU The motion capture technology is widely used and has achieved greater market in virtual reality and animation game industry. The application of motion capture technology can solve the bottleneck problem of defining the motion trajectory of virtual role in the 3D animation, and decrease the workload of controlling virtual roles, and consequently increase the efficiency of game development.

The mainstream motion capture systems are the optical and mechanical sensors [11]. The optical system needs too many high-speed cameras, so the price is higher and the workload of post-data processing is larger. The mechanical one has no specific requirement to the space, but good real time and high capture precision. The system can be used not only in 3D animation, but also in virtual reality, virtual training and simulation, more generally, nearly all kinds of motion measurement of animals or other objects.

Zhang and Wu developed a hierarchical sensor fiision method for the estimation of upper limb motions from IMU signals [12]. Their algorithm consists of a sensor data de-noising and fusion level and a geometrical constraint fusion level. Kalman filtering is also commonly used for the fusion of IMU signals. The sensor fusion algorithms for an IMU are challenged by the calibration problem and the frequency-dependent reliability of each measurement. Frosio et al. proposed an automated calibration method for MEMS accelerometers, which is based on a fact that the accelerometer should read the gravitational acceleration in a stationary state [13]. Even if the accelerometers are calibrated accurately, however, the attitude estimation by accelerometers is still sensitive to motion accelerations. For example, when the magnitude of motion accelerations is small, the accelerometers may result in a very accurate estimation of the attitude, but the accuracy would be drastically lowered when the motion acceleration is large. On the other hand, the gyroscope measurements suffer from a drift problem, because they are always to be integrated to obtain the attitude information. To avoid this drift problem, gyroscopes are often highpass-filtered and combined with the estimation by accelerometers. Zhou and Hu reported that the attitude estimation performance can be dramatically improved when the gyroscope and accelerometer measurements are effectively fused [14]. The IMU based MCS may work well with the sensor fusion, it still has the reference point issue. Since the IMU estimates the orientation of each body segment only, a reference point (i.e., a stationary point that provides the absolute position and orientation information) must be given for the construction of the whole body motion. There are a few previously proposed and/or commercialized MCSs based on IMUs [15, 16]. Most of these systems construct the whole body motion from the IMU measurements with a fixed reference point [17]. When the motion of the reference body segment is large, however, the calculated whole body motion is not an accurate representation of the actual human motion. Moreover, when the human moves a long distance in an open space, it is difficult to capture the motions by the existing methods.

In this paper, a human motion capture system based on IMUs is proposed and the development process of mechanical human body motion capture system based on inertial sensors is described. The system mainly involves human body motion capture model construction, IMU technology, Bluetooth, sensor network and software development.

3. Application Development of MCS This human MCS is developed based on sensor technology, wireless communication technology and 3D animation software SDK. It can be divided into sensor network, master control module, wireless data transmission module and real-time graphical display software of the amateur human body.

When a human MCS is developed, a human body motion must be abstracted and simplified into a human skeleton model based on his principles of kinesiology and then its regular exercise and changes are studied. The system discussed in this paper divides human body into 17 key joint points, and inertial sensors are set on the corresponding key points. All of the 17 sensors constitute a sensor network. The sensor network is used to obtain the real-time motion data of key joint points of the human body.

Each inertial sensor consists of three internal sensors such as tri-axial acceleration sensor, tri-axial angular velocity sensor and tri-axis magnetic sensor, and MCU.

The inertial sensor used in this system applies Kalman filter algorithm to integrate and analyze the raw measurement data of internal three sensors. The inertial sensor outputs the motion data in the form of Euler angle or Quaternion, and the data are translated from the inertial sensors to the reconstruction software of the human body motion model on the workstation through Bluetooth, and the software analyzes and calculates the relative displacement of key joints through analytic inverse kinematics algorithm [18], and displaying the real-time human body motion posture graphically. At the same time the software can output storage file with the suffix .BVH (Biovision Hierarchy), thus the human body motion capture system realizes the real-time reconstruction of human body motion model. The principle of human body motion capture system is shown in Fig. 1. Each box in the diagram represents a hardware part of the system. Each sensor in the sensor network is connected to the master controller module through RS485. When two master controller modules are sending data in real-time, at the same time, the two corresponding receiving end in the workstation are receiving data with a wireless way in real-time.

According to the way of processing motion data the human body MCS can be divided into two modules, the data acquisition module and the data processing module.

Data acquisition module mainly collects the rotation angle data of 17 key joints through sensors network, and it is the basis of the real-time reconstruction of human body motion model. The data processing module is used to process the measurement data of the inertial sensor of each joint point, and retrieve the performer's motion posture data, and then the system software graphically displays the real-time motion posture of the virtual role corresponding to the true performer.

The system is divided into modules based on data flow, and the function of the system implementation is obtained based on accurate motion data. The data here refers to the movement data of each key of joint points of the human body, and these data come originally from the sensor nodes in the sensor network and finally flow into the graphics software, and at last the data are converted into the real-time image by the graphics software. The captured human motion pose data are saved as .FBX file, which is a powerful and strong commonality of data file format.

Fig. 2 clearly describes the direction of system data flow and the process of data flow. The figure shows that the system can be divided into the following several important modules: the sensor network, the master controller module, the wireless data transmission module and the graphics software of real-time reconstruction for human body model.

3.1. Data Acquisition Module 3.1.1. Design of Sensor Network Node The sensor network is mainly used to obtain motion data of the key joints of human body. In this paper, the system abstracts the human body motion into the movement of the human skeleton model, and human skeleton is a joint chain structure which is composed by a series of the sequentially connected rigid bodies, and here the junction of two rigid bodies is so called joint. So the human skeleton is divided into 17 critical joints, and the whole-body motion posture is built through integration and calculation of the motion data of 17 joints. Therefore, the motion of the whole body can be viewed as the motion of human skeleton model which is connected by 17 critical joints, and hip abdominal joint is taken as the root of joints to determine world space position and orientation of human body. Here stratified skeleton structure (i.e., tree structure) is used for graphic description, as shown in Fig. 3.

The process to build hierarchical model of human body joints is a procedure to extract key joint points of human body because the motion data of key joint points can reflect the motion posture of the whole human body after the motion data are processed. In order to obtain the motion data of 17 key joint points, the system places 17 inertial sensors in the corresponding key points, so the sensor network of the application contains 17 sensor nodes and each sensor gets motion data of the specific joint point. In the reconstruction software of human body motion model, the motion data of 17 key joint points are analyzed and calculated through analytical inverse kinematics algorithm, then the relative displacement of each joint point is obtained by the software, and the true performer's motion posture is reconstructed at last.

Therefore, the distribution of sensor network nodes is determined by the hierarchical model of human body joint, as shown in Fig. 4, and each black point represents a sensor network node in the diagram. MEMS technology (Micro-ElectroMechanical Systems) has made great progress now, and the measurement accuracy and internal integration of various sensing unit is more and more high. In this system inertial sensors are used in the sensor network to capture the real-time motion data of the key joints, and then integrate and calculates the measurement data of internal three sensors through the Kalman filter algorithm [19]. Another feature of the inertial sensor is the internal integration of the three types of sensor units, and the measurement data of the three kinds of sensors compensate each other, so the inertial sensor can measure the optimum motion data. The function of inertial sensor in the system is shown in Fig. 2.

Inertial sensor also has the parameter adjustment function. If a ferromagnetic substance exists in the working environment or field magnetic intensity of working environment is unstable, user can try to reduce the sensitivity of the magnetic sensor, and relatively increase the sensitivity of angular velocity sensor and acceleration sensor to adapt to the environment changing.

3.1.2. Design of Master Controller System There are two master controllers in the motion capture system, which respectively obtain the right and left bust motion data from sensor network nodes. The master controller utilizes the ARM chip STM32F103RE. It well meets the requirements of real-time data acquisition and transmission of the system. Each sensor network node connects to master controller with RS-485 bus of a higher data transfer rate (10 Mbps). The interface of the bus also has advantages of good anti-noise and longer transmission distance.

The Bluetooth transmission module is integrated in the master controller system, which is used to wirelessly send motion data of key joints to the workstation (PC). So the master controller system can transmit the real-time motion data to the reconstruction software of human body motion model on the workstation. The motion data of left half body are obtained by corresponding sensors and sent to the master controller for left half body based on RS485 data communication. The right master controller does the something for right half body by the same way. The data processing procedure is shown as in Fig. 5.

3.1.3. Design of Bluetooth Transceiver The wireless data transmission module is one of the key contents of human MCS and Bluetooth is applied in this paper. Bluetooth technology is an open global specification of wireless data and voice communication, which based on low-cost and lowpower wireless connection [20]. Bluetooth transmission module of the system discussed in this paper uses Bluetooth chip AUDIO-FLASH made by the CRS company, and adds Atmel T7024 Bluetooth chip which dedicated to front-end chip to be expand into standard class 1 Bluetooth module. It is one of innovative technologies of the human body motion capture system to apply Bluetooth for realization of wireless transmission of the motion data. The wireless data transmission between master controllers and the workstation is done by sender and receiver of Bluetooth transmission modules integrated respectively both in the master controller and workstation. Fig. 6 shows the main functions and hardware relationship of human MCS.

3.2. Data Processing The motion data processing is completed by the software of motion capture system. The reconstruction software of human body motion posture should have the functions of intelligent data processing and real-time reconstruction of the motion posture, and at the same time it can also edit and modify the recorded file. The development of the software is based on the integrated development environment of Visual C++ 6.0, completed with C++ programming language and the techniques of MSComm communication controls, MFC, COM components, and OpenGL. The software of human body motion capture involves two parts, the motion posture display module and system configuration module. The system configuration module communicates with Bluetooth devices of specified ports, and obtains the motion data of 17 key points of the whole body. Communication between the reconstruction software of motion model and Bluetooth devices is realized through MSComm controls and the serial ports. MSComm control provides the OnComm events and CommEvent attribute to capture and monitor the values of communication events, and functions of Input and Output can be used to read or write character data to the corresponding buffers. The motion posture data collected from Bluetooth module is delivered to the motion posture display module of the software, which is mainly used to display performer's real-time posture in three-dimensional scene, and then the synchronization of motion posture of the virtual role and true performer is realized. The accuracy of motion capture depends on the algorithm to calculate the human body motion data, namely it depends on that how to use the data of each sensor for getting the performer's motion posture. The system analyzes and calculates the motion data through analytical inverse kinematic algorithm, which can effectively improve the calculation speed and real-time characteristics of the system [21]. Inverse kinematics algorithm uses the formulae directly to calculate the motion posture data of middle joints of the joint chain through the known location and status of the end effector, which consequently improve the efficiency of motion capture. Fig. 7 shows the motion posture that the performer is waving left hand. Fig. 8 shows the amateur human motion posture versus graphics display.

4. Conclusions The expansion in motion capture use may be driven by the needs and relative cost effectiveness of non-optical technologies. Optical systems, while achieving higher measurement accuracy, are often limited to fixed laboratory spaces. The portability of an inertial system leads to automation applications that are less constrained by the environment and could lead to greater deployment in industry.

The system as an example of development in this paper has three features. Firstly, it collects the motion data for the key joint points of human body through inertial sensor network. Secondly, the system uses the Bluetooth technology to realize the wireless transmission of the human body motion data, which effectively improves the flexibility of the actions of the performers. Thirdly, since each of sensors of the system has a higher relative independence, it is convenient to expand the study of the motion capture system, such as the research of multiplayer real-time motion capture technology, which will be the next step to do the further research.

References [1]. L. M. Steffen, D. K. Arnett, H. Blackburn, G. Shah, C. Armstrong, R. V. Luepker, D. R. Jacobs, Population trends in leisure-time physical activity: Minnesota hear survey, 1980-2000, Medicine and Science in Sports and Exercise, Vol. 38, No. 10, 2006, pp. 1716-1723.

[2]. A. Y.-L. Chong, F. T. S. Chan, Structural Equation Modeling for Multi-Stage Analysis on Radio Frequency Identification (RFID) Diffusion in the Health Care Industry, Expert Systems with Applications, Vol. 39, No. 10,2012, pp. 8645-8654.

[3]. E. Zetou, V. Koronas, I. Athanailidis, P. Koussis, Learning tennis skill through game play and stay in elementary pupils, Journal of Human Sport and Exercise, Vol. 7, No. 2,2012, pp. 560-572.

[4]. J. C. P. Chan, H. Leung, J. K. T. Tang, T. Komura, A virtual reality dance training system using motion capture technology, IEEE Transactions on Learning Technologies, Vol. 4, Issue 2,2011, pp. 187-195.

[5]. K. Kong, M. Tomizuka, A gait monitoring system based on air pressure sensors embedded in a shoe, IEEE/ASME Transactions on Mechatronics, Vol. 14, Issue 3,2009, pp. 358-370.

[6]. S. Hashi, Y. Tokunaga, S. Yabukami, M. Toyoda, K. Ishiyama, Y. Okazaki, K. I. Arai, Development of real-time and highly accurate wireless motion capture system utilizing soft magnetic core, IEEE Transactions on Magnetics, Vol. 41, Issue 10, 2005, pp. 4191-4193.

[7]. S. Kwon, H.-S. Park, C. J. Stanley, J. Kim, J.H. Kim, D. L. Damiano, A practical strategy for EMG-based knee joint moment estimation during gait and its validation in individuals with cerebral palsy, IEEE Transactions on Biomedical Engineering, Vol. 59, Issue 5,2012, pp. 1480-1487.

[8]. A. G. Cutti, A. Giovanardi, L. Rocchi and A. Davalli, A simple test to assess the static and dynamic accuracy of an inertial sensors system for human movement analysis, in Proceedings of the 28,h IEEE EMBS Annual International Conference, New York, NY, USA, August 30-September 3, 2006, pp. 1002-1009.

[9]. D. Roetenberg, H. J. Luinge, C. T. M. Baten and P. H. Veltink, Compensation of magnetic disturbances improves inertial and magnetic sensing of human body segment orientation, IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 13, Issue 3, 2005, pp. 395-405.

[10]. D. Vlasic, R. Adelsberger, G. Vannucci, J. Barnwell, M. Gross, W. Matusik and J. Popovic, Practical motion capture in everyday surroundings, ACM Transactions on Graphics, Vol. 26, No. 3, 2007, pp. 35:1-35:9.

[11]. Huang Boshi, Chen Fumin, Research on human body motion capture and motion control, Computer Engineering and Applications, Vol. 41, No. 7, 2005, pp. 60-63.

[12]. Z.-Q. Zhang, J.-K. Wu, A novel hierarchical information fusion method for three-dimensional upper limb motion estimation, IEEE Transactions on Instrumentations and Measurement, Vol. 60, Issue 11,2011, pp. 3709-3719.

[13]. I. Frosio, F. Pedersini, N. A. Borghese, Autocalibration of MEMS accelerometers, IEEE Transactions on Instrumentations and Measurement, Vol. 58, No. 6,2009, pp. 2034-2041.

[14]. H. Zhou, H. Hu, Reducing drifts in the inertial measurements of wrist and elbow positions, IEEE Transactions on Instrumentations and Measurement, Vol. 59, No. 3,2010, pp. 575-585.

[15]. N. Miller, O. C. Jenkins, M. Kallmann, M. J. Mataric, Motion capture from inertial sensing for untethered humanoid teleoperation, in Proceedings of the IEEE International Conference on Humanoid Robots, Vol. 2,2004, pp. 547-565.

[16]. M. Brodie, A. Walmsley, W. Page, Fusion motion capture: a prototype system using inertial measurement units and GPS for the biomechanical analysis of ski racing, Sports Technologies, Vol. 1, No. 1,2008, pp. 17-28.

[17]. Z. Lin, M. Zecca, S. Sessa, L. Bartolomeo, H. Ishii, K. Itoh, Development of an ultra-miniaturized inertial measurement unit WB-3 for human body motion tracking, in Proceedings of the IEEE International Symposium on System Integration, 2010, pp. 414-419.

[18]. Anders Lau Olsen, Henrik Gordon Petersen, Inverse kinematics by numerical and analytical cyclic coordinate descent, Robotica, Vol. 29, No. 4, 2011, pp. 619-626.

[19]. Cao Chun-Ping, Luo Ling-Li, Wireless indoor positioning system based on Kalman filtering algorithm, Computer Systems & Applications, Vol. 20, No. 11, 2011,pp. 76-79.

[20]. You Ke, Liu Ruiqiang, Bluetooth-based wireless sensor networks, in Proceedings of the Second International Conference on Space Information Technology, Wuhan, China, 2007.

[21]. Wang Yi-Gang, Wu Zi-Zhao, Chen Xiao-Diao, Kind of analytical inverse kinematics method, Application Research of Computers, Vol. 26, No. 6, 2009, pp. 2368-2369.

Guangtian Shi, Yongsheng Wang, Shuai Li Lanzhou Jiaotong University, Lanzhou, China Tel: 086-931-4938013 E-mail: [email protected] Received: 18 April 2014 /Accepted: 30 May 2014 /Published: 30 June 2014 (c) 2014 IFSA Publishing, S.L.

[ Back To TMCnet.com's Homepage ]