In this analysis, we present results from measurements performed to determine the stability of a hand tracking system and the accuracy of the detected palm and finger's position. Measurements were performed for the evaluation of the sensor for an application in an industrial robot-assisted assembly scenario. Human-robot interaction is a relevant topic in collaborative robotics. Intuitive and straightforward control tools for robot navigation and program flow control are essential for effective utilisation in production scenarios without unnecessary slowdowns caused by the operator. For the hand tracking and gesture-based control, it is necessary to know the sensor's accuracy. For gesture recognition with a moving target, the sensor must provide stable tracking results. This paper evaluates the sensor's real-world performance by measuring the localisation deviations of the hand being tracked as it moves in the workspace.

Department of Robotics Faculty of Mechanical Engineering Technical University of Kosice 04200 Kosice Slovakia

Department of Robotics Faculty of Mechanical Engineering VSB Technical University of Ostrava 70800 Ostrava Czech Republic

See more in PubMed

Sheridan T.B. Human-Robot Interaction: Status and Challenges. Hum. Factors. 2016;58:525–532. doi: 10.1177/0018720816644364. PubMed DOI

Chen B., Hua C., Dai B., He Y., Han J. Online control programming algorithm for human-robot interaction system with a novel real-time human gesture recognition method. Int. J. Adv. Robot. Syst. 2019;16 doi: 10.1177/1729881419861764. DOI

Maurtua I., Fernández I., Tellaeche A., Kildal J., Susperregi L., Ibarguren A., Sierra B. Natural multimodal communication for human-robot collaboration. Int. J. Adv. Robot. Syst. 2017 doi: 10.1177/1729881417716043. DOI

Aliprantis J., Konstantakis M., Nikopoulou R., Mylonas P., Caridakis G. Natural Interaction in Augmented Reality Context; Proceedings of the Visual Pattern Extraction and Recognition for Cultural Heritage Understanding; Pisa, Italy. 30 January 2019; pp. 50–61.

Bachmann D., Weichert F., Rinkenauer G. Review of Three-Dimensional Human-Computer Interaction with Focus on the Leap Motion Controller. Sensors. 2018;18:2194. doi: 10.3390/s18072194. PubMed DOI PMC

Leap Motion Coordinate Systems. [(accessed on 16 June 2020)]; Available online: https://developer-archive.leapmotion.com/documentation/objc/devguide/Leap_Coordinate_Mapping.html.

Vysocký A., Pastor R., Novák P. Interaction with Collaborative Robot Using 2D and TOF Camera; Proceedings of the Modelling and Simulation for Autonomous Systems (MESAS 2018); Prague, Czech Republic. 17–19 October 2018; pp. 477–489.

Bachmann D., Weichert F., Rinkenauer G. Evaluation of the Leap Motion Controller as a New Contact-Free Pointing Device. Sensors. 2015;15:214–233. doi: 10.3390/s150100214. PubMed DOI PMC

Guna J., Jakus G., Pogačnik M., Tomažič S., Sodnik J. An Analysis of the Precision and Reliability of the Leap Motion Sensor and Its Suitability for Static and Dynamic Tracking. Sensors. 2014;14:3702–3720. doi: 10.3390/s140203702. PubMed DOI PMC

Smeragliuolo A.H., Hill N.J., Disla L., Putrino D. Validation of the Leap Motion Controller Using Markered Motion Capture Technology. J. Biomech. 2016;49:1742–1750. doi: 10.1016/j.jbiomech.2016.04.006. PubMed DOI

Vosinakis S., Koutsabasis P. Evaluation of Visual Feedback Techniques for Virtual Grasping with Bare Hands using Leap Motion and Oculus Rift. Virtual Real. 2017 doi: 10.1007/s10055-017-0313-4. DOI

Oropesa I., de Jong T., Sánchez-González P., Dankelman J., Gómez E. Feasibility of tracking laparoscopic instruments in a box trainer using a Leap Motion Controller. Measurement. 2016;80:115–124. doi: 10.1016/j.measurement.2015.11.018. DOI

Pititeeraphab Y., Choitkunnan P., Thongpance N., Kullathum K., Pintavirooj C. Robot-arm control system using LEAP motion controller; Proceedings of the 2016 International Conference on Biomedical Engineering (BME-HUST); Hanoi, Vietnam. 5–6 October 2016; pp. 109–112.

Gunawardane H., Medagedara N., Madhusanka B. Control of Robot Arm Based on Hand Gesture using Leap Motion Sensor Technology. Int. J. Robot. Mechatronics. 2017 doi: 10.21535/ijrm.v3i1.930. DOI

Staretu I., Moldovan C. Leap Motion Device Used to Control a Real Anthropomorphic Gripper. Int. J. Adv. Robot. Syst. 2016;13:113. doi: 10.5772/63973. DOI

Jin H., Chen Q., Chen Z., Hu Y., Zhang J. Multi-LeapMotion sensor based demonstration for robotic refine tabletop object manipulation task. Caai Trans. Intell. Technol. 2016;1:104–113. doi: 10.1016/j.trit.2016.03.010. DOI

Hand Gesture Interface for Robot Path Definition in Collaborative Applications: Implementation and Comparative Study