RT info:eu-repo/semantics/article
T1 One-shot learning for rapid generation of structured robotic manipulation tasks from 3D video demonstrations
A1 Duque Domingo, Jaime
A1 Caccavale, Riccardo
A1 Finzi, Alberto
A1 Zalama Casanova, Eduardo
A1 Gómez García-Bermejo, Jaime
K1 Aprendizaje único
K1 Aprendizaje robótico
K1 Aprendizaje por demostración
K1 Segmentación de actividades
K1 1203 Ciencia de Los Ordenadores
K1 1203.04 Inteligencia Artificial
AB We present a framework that enables a collaborative robot to rapidly replicate structured manipulation tasks demonstrated by a human operator through a single 3D video recording. The system combines object segmentation with hand and gaze tracking to analyze and interpret the video demonstrations. The manipulation task is decomposed into primitive actions that leverage 3D features, including the proximity of the hand trajectory to objects, the speed of the trajectory, and the user’s gaze. In line with the One-Shot Learning paradigm, we introduce a novel object segmentation method called SAM+CP-CVV, ensuring that objects appearing in the demonstration require labeling only once. Segmented manipulation primitives are also associated with object-related data, facilitating the implementation of the corresponding robotic actions. Once these action primitives are extracted and recorded, they can be recombined to generate a structured robotic task ready for execution. This framework is particularly well-suited for flexible manufacturing environments, where operators can rapidly and incrementally instruct collaborative robots through video-demonstrated tasks. We discuss the approach applied to heterogeneous manipulation tasks and show that the proposed method can be transferred to different types of robots and manipulation scenarios.
PB Springer Nature
SN 0956-5515
YR 2025
FD 2025
LK https://uvadoc.uva.es/handle/10324/79353
UL https://uvadoc.uva.es/handle/10324/79353
LA eng
NO Journal of Intelligent Manufacturing, 2025.
NO Producción Científica
DS UVaDOC
RD 10-ene-2026