Por favor, use este identificador para citar o enlazar este ítem:https://uvadoc.uva.es/handle/10324/61780
Título
Lightweight real-time hand segmentation leveraging MediaPipe landmark detection
Autor
Año del Documento
2023
Editorial
Springer
Descripción
Producción Científica
Documento Fuente
Virtual Reality, 2023.
Resumo
Real-time hand segmentation is a key process in applications that require human–computer interaction, such as gesture rec-
ognition or augmented reality systems. However, the infinite shapes and orientations that hands can adopt, their variability
in skin pigmentation and the self-occlusions that continuously appear in images make hand segmentation a truly complex
problem, especially with uncontrolled lighting conditions and backgrounds. The development of robust, real-time hand
segmentation algorithms is essential to achieve immersive augmented reality and mixed reality experiences by correctly
interpreting collisions and occlusions. In this paper, we present a simple but powerful algorithm based on the MediaPipe
Hands solution, a highly optimized neural network. The algorithm processes the landmarks provided by MediaPipe using
morphological and logical operators to obtain the masks that allow dynamic updating of the skin color model. Different
experiments were carried out comparing the influence of the color space on skin segmentation, with the CIELab color space
chosen as the best option. An average intersection over union of 0.869 was achieved on the demanding Ego2Hands dataset
running at 90 frames per second on a conventional computer without any hardware acceleration. Finally, the proposed seg-
mentation procedure was implemented in an augmented reality application to add hand occlusion for improved user immer-
sion. An open-source implementation of the algorithm is publicly available at https://github.com/itap-robotica-medica/light
weight-hand-segmentation.
Materias Unesco
33 Ciencias Tecnológicas
Palabras Clave
Augmented reality
Hand segmentation
MediaPipe
Online processing
Semantic segmentation
ISSN
1359-4338
Revisión por pares
SI
Patrocinador
Ministerio de Ciencia e Innovación (under Grant Agreement No. RTC2019-007350-1)
Publicación en abierto financiada por el Consorcio de Bibliotecas Universitarias de Castilla y León (BUCLE), con cargo al Programa Operativo 2014ES16RFOP009 FEDER 2014-2020 DE CASTILLA Y LEÓN, Actuación:20007-CL - Apoyo Consorcio BUCLE
Publicación en abierto financiada por el Consorcio de Bibliotecas Universitarias de Castilla y León (BUCLE), con cargo al Programa Operativo 2014ES16RFOP009 FEDER 2014-2020 DE CASTILLA Y LEÓN, Actuación:20007-CL - Apoyo Consorcio BUCLE
Version del Editor
Propietario de los Derechos
© 2023 The Author(s)
Idioma
eng
Tipo de versión
info:eu-repo/semantics/publishedVersion
Derechos
openAccess
Aparece en las colecciones
Arquivos deste item
Exceto quando indicado o contrário, a licença deste item é descrito como Atribución 4.0 Internacional