Although the field of sleep study has greatly developed over recent years, the most common and efficient way to detect sleep issues remains a sleep examination performed in a sleep laboratory. This examination measures several vital signals by polysomnograph during a full night's sleep using multiple sensors connected to the patient's body. Nevertheless, despite being the gold standard, the sensors and the unfamiliar environment's connection inevitably impact the quality of the patient's sleep and the examination itself. Therefore, with the novel development of accurate and affordable 3D sensing devices, new approaches for non-contact sleep study have emerged. These methods utilize different techniques to extract the same breathing parameters but with contactless methods. However, to enable reliable remote extraction, these methods require accurate identification of the basic region of interest (ROI), i.e., the patient's chest area. The lack of automated ROI segmenting of 3D time series is currently holding back the development process. We propose an automatic chest area segmentation algorithm that given a time series of 3D frames containing a sleeping patient as input outputs a segmentation image with the pixels that correspond to the chest area. Beyond significantly speeding up the development process of the non-contact methods, accurate automatic segmentation can enable a more precise feature extraction. In addition, further tests of the algorithm on existing data demonstrate its ability to improve the sensitivity of a prior solution that uses manual ROI selection. The approach is on average 46.9% more sensitive with a maximal improvement of 220% when compared to manual ROI. All mentioned can pave the way for placing non-contact algorithms as leading candidates to replace existing traditional methods used today.

Department of Computing and Control Engineering University of Chemistry and Technology Prague Czech Republic

Department of Mathematics and Computer Science Open University of Israel Raanana Israel

See more in PubMed

Flemons WW, Littner MR, Rowley JA, Gay P, Anderson WM, Hudgel DW, McEvoy RD, Loube DI (2003) Home diagnosis of sleep apnea: a systematic review of the literature. Chest 124(4):1543–1579 [Online]. Available: http://linkinghub.elsevier.com/retrieve/pii/S0012369216487052

Procházka A, Schätz M, Centonze F, Kuchyňka J, Vyšata O, Vališ M (2016) Extraction of breathing features using MS Kinect for sleep stage detection. Signal Image and Video Processing 10(7):1279–1286 DOI

Gibson, GJ (2004) Obstructive sleep apnoea syndrome: underestimated and undertreated. Br Med Bull 72(1):49–64. [Online]. Available: https://academic.oup.com/bmb/article/72/1/49/272854

Al-Naji A, Gibson K, Lee S-H, Chahl J (2017) Real time apnoea monitoring of children using the microsoft kinect sensor: a pilot study. Sensors 17(2):286. [Online]. Available: http://www.mdpi.com/1424-8220/17/2/286

Palmero C, Esquirol J, Bayo V, Cos MÀ, Ahmadmonfared P, Salabert J, Sánchez D, Escalera S (2017) Automatic sleep system recommendation by multi-modal RBG-depth-pressure anthropometric analysis. Int J Comput Vision 122(2):212–227. [Online]. Available: http://link.springer.com/10.1007/s11263-016-0919-0

Schätz M, Procházka A, Kuchyňka J, Vyšata O (2020) Sleep apnea detection with polysomnography and depth sensors. Sensors (Switzerland)

Hirshkowitz M (2014) Polysomnography: understanding this technology’s past might guide future developments. IEEE Pulse 5(5):26–28. [Online]. Available: http://ieeexplore.ieee.org/document/6908144/

Young T, Peppard PE, Gottlieb DJ (2002) Epidemiology of obstructive sleep apnea: a population health perspective, pp 1217–1239

Charlton PH, Birrenkott DA, Bonnici T, Pimentel MAF, Johnson AEW, Alastruey J, Tarassenko L, Watkinson PJ, Beale R, Clifton DA (2018) Breathing rate estimation from the electrocardiogram and photoplethysmogram: a review. IEEE Rev Biomed Eng 11:2–20. [Online]. Available: https://doi.org/10.1109/rbme.2017.2763681

Lazazzera R, Laguna P, Gil E, Carrault G (2021) Proposal for a home sleep monitoring platform employing a smart glove. Sensors 21(23). [Online]. Available: https://www.mdpi.com/1424-8220/21/23/7976

Dafna E, Tarasiuk A, Zigel Y (2015) Sleep-wake evaluation from whole-night non-contact audio recordings of breathing sounds. PLoS ONE 10(2)

Harte JM, Golby CK, Acosta J, Nash EF, Kiraci E, Williams MA, Arvanitis TN, Naidu B (2016) Chest wall motion analysis in healthy volunteers and adults with cystic fibrosis using a novel Kinect-based motion tracking system, Med Biol Eng Comput 54(11):1631–1640. [Online]. Available: http://link.springer.com/10.1007/s11517-015-1433-1

Kagawa M, Ueki K, Tojima H, Matsui T (2013) Noncontact screening system with two microwave radars for the diagnosis of sleep apnea-hypopnea syndrome, Conference proceedings: ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual Conference 2013:2052–2055

Yang C, Cheung G, Stankovic V, Chan K, Ono N (2017) Sleep apnea detection via depth video and audio feature learning. IEEE Trans Multimed 19(4):822–835 DOI

Schätz M, Centonze F, Kuchynka J, Tupa O, Vysata O, Geman O, Prochazka A (2015) Statistical recognition of breathing by MS Kinect depth sensor. In: 2015 International workshop on computational intelligence for multimedia understanding (IWCIM). IEEE, pp 1–4. [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=7347062

Nguyen P, Transue S, Choi M-H, Halbower AC, Vu T (2016) WiKiSpiro: non-contact respiration volume monitoring during sleep. 8th Wireless of the Students, by the Students, and for the Students Workshop, S3, vol. 03-07-Octo, no 2, pp. 27–29. [Online]. Available: https://www.scopus.com/inward/record.uri?eid=2-s2.0-84994184650&partnerID=40&md5=5f9db30d9f6744108d158e859642a55e

Carfagni M, Furferi R, Governi L, Servi M, Uccheddu F, Volpe Y (2017) On the performance of the intel sr300 depth camera: metrological and critical characterization. IEEE Sensors J 17(14):4508–4519 DOI

Schwarz LA, Mkhitaryan A, Mateus D, Navab N (2012) Human skeleton tracking from depth data using geodesic distances and optical flow. Image and Vision Computing 30(3):217–226 DOI

Garn H, Kohn B, Dittrich K, Wiesmeyr C, Kloesch G, Stepansky R, Wimmer M, Ipsiroglu O, Grossegger D, Kemethofer M, Seidel S (2016) 3D detection of periodic limb movements in sleep. 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 427–430. [Online]. Available: http://ieeexplore.ieee.org/document/7590731/

Chen L, Wei H, Ferryman J (2013) A survey of human motion analysis using depth imagery. Pattern Recogn Lett 34(15):1995–2006 DOI

Schwarz LA, Mkhitaryan A, Mateus D, Navab N (2011) Estimating human 3D pose from Time-of-Flight images based on geodesic distances and optical flow. 2011 IEEE international conference on automatic face and gesture recognition and workshops, FG 2011, pp 700–706

Cao Z, Hidalgo G, Simon T, Wei SE, Sheikh Y (2021) OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 43(1):172–186 DOI

Wang W, Zhu H, Dai J, Pang Y, Shen J, Shao L (2020) Hierarchical human parsing with typed part-relation reasoning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 8929–8939

Wang W, Zhang Z, Qi S, Shen J, Pang Y, Shao L (2019) Learning compositional neural information fusion for human parsing. In: 2019 IEEE/CVF international conference on computer vision (ICCV). pp 5702–5712

Plagemann C, Ganapathi V, Koller D, Thrun S (2010) Real-time identification and localization of body parts from depth images. In: 2010 IEEE international conference on robotics and automation. IEEE, pp 3108–3113

Shotton J, Fitzgibbon A, Cook M, Sharp T, Finocchio M, Moore R, Kipman A, Blake A (2011) Real-time human pose recognition in parts from single depth images. In CVPR 2011. IEEE, pp 1297–1304

Okuno A, Ishikawa T, Watanabe H (2020) Rollover detection of infants using posture estimation model. In: 2020 IEEE 9th global conference on consumer electronics (GCCE). IEEE, pp 490–493

Mallick T, Das PP, Majumdar AK (2014) Characterizations of noise in Kinect depth images: a review. pp. 1731–1740

Huang W, Wai AAP, Foo SF, Biswas J, Hsia C-C, Liou K (2010) Multimodal sleeping posture classification. In: 2010 20th International conference on pattern recognition. IEEE, pp 4336–4339