TMCnet News

uSens Debuts World's First AI-based 3D Skeletal Hand Tracking SDK for Smartphones
[May 31, 2018]

uSens Debuts World's First AI-based 3D Skeletal Hand Tracking SDK for Smartphones


Augmented World Expo - With the goal of making it easy for everyone to experience augmented reality (AR), uSens, Inc., a pioneer in HCI and computer-vision tracking solutions for augmented and virtual reality (VR), today announced the beta launch of its uSensAR Hand Tracking SDK.

An innovative, new AR/VR positional development tool, the uSens AR Hand Tracking SDK empowers developers to create AI and skeletal hand tracking content for the smartphone, including both iOS and android, using the camera on the phone- bringing an enhanced AR experience to the masses. uSens will demo this new technology at Augmented World Expo 2018 (AWE 2018) in Santa Clara, May 31 - June 1.

"While Pokémon Go and Snapchat brought augmented reality mainstream and into the lives of hundreds and millions of smartphone users, these interactions have so far been limited to their touchscreens," said Anli He, co-founder and CEO of uSens. "uSens is proud to push AR to the next frontier, by enabling developers to create engaging, enjoyable and entertaining augmented reality experiences made more intuitive, for the smartphone user-by simply moving their hands and fingers in the air."

"This opens a whole new world of possibilities for developers, enabling them to create a truly one-of-kind experience for a mainstream audience. Similar to how touchscreens enabled even the most technologically challenged to embrace smartphones, providing an easy and natural way for users to engage with AR/VR objects and environments will play a major role in boosting consumer adoption," said He.

Unlike other technologies that only track fingertips, uSensAR Hand Tracking SDK uses computer vision and deep learning technology with a mobile phone's RGB camera to provide full hand skeleton tracking and 3D motion recognition.

uSens' hand skeleton and 3D-motin recognition software:



  • Users only need to move their hands in the air to place themselves in the virtual world.
  • Provides 3D skeletal hand tracking, allowing for precise, controller-less interaction on an AR/VR device.
  • Greatly reduces reliance on external input devices such as game pads or external cameras, providing more intuitive control in mobile devices.
  • Is compatible with both Android (News - Alert) and iOS systems.

uSens' innovative hand skeleton and 3D-motion recognition technology opens up a whole new world of potential target markets not possible with current virtual reality products - such as virtual training and education, healthcare/medical, entertainment and creating new emotional connections through shared virtual experiences.

The new software makes it simple for developers and manufacturers to add AR/VR capabilities. Applications can be created based on the system to serve almost all mobile users.


Partner with uSens

uSensAR Hand Tracking SDK is currently in beta with several major companies including a 270-million plus user top-ranked social networking app, and a major gaming firm. Interested partners can contact uSens at [email protected]

Visit uSens at Augmented World Expo (AWE)

From May 31 to June 1, uSens will be exhibiting at Booth #930 at AWE held at the Santa Clara Convention Center. Attendees can try first-hand uSens' various demos of our new hand skeleton and 3D motion recognition software for AR/VR.

About uSens, Inc.

uSens is a pioneer in 3D HCI technology that is changing how we interact with next-generation technology forever. Our team possesses decades of domain experience in computer vision and artificial intelligence acquired at companies like Apple, Samsung (News - Alert) and Intel. We provide augmented & virtual reality, mobile phone, automotive, and IoT companies with the most natural, intuitive and immersive interactive solutions.

Our pioneering inside-out, 26DOF (degrees of freedom) hand tracking technology is a breakthrough in HCI allowing users to control virtual worlds with their bare hands. Our inside-out 6DOF positional tracking solution based on SLAM technology is a key component in building immersive AR/VR experiences. Founded in 2013 and headquartered in Silicon Valley with additional offices in China, uSens has raised $26.6M in VC funding to date. Contact us at [email protected], visit us at www.usens.com, and follow us at @usensinc.

Editor's notes

3D skeletal Hand Tracking SDK video can be found here


[ Back To TMCnet.com's Homepage ]