Yamakawa Laboratory, The University of Tokyo

東京大学　山川研究室

In recent years, the realization of a society where humans and robots coexist has become highly anticipated. As a result, robots must possess versatility, independent of their operating environment or target objects, along with the ability to operate with high responsiveness to ensure safety and handle dynamic tasks. To achieve this, we addressed cooperative catching in close proximity to humans and designed a comprehensive system consisting of three main components: recognition, robot target estimation, and motion planning. For recognition, we implemented a SORT-based multi-object tracking (MOT), stereo-cameraspecific geometry-based inter-camera matching method and a high-speed skeleton tracking technique, combining deep learning-based detection with optical flow-based motion extraction. For target selection, we developed a cost-function-based approach to determine the optimal catching target and its position. For motion planning, we introduced a high-speed path planning model designed to address four key challenges: (P1) avoiding local minima, (P2) suppressing oscillations, (P3) ensuring applicability to dynamic environments, and (P4) handling obstacles with arbitrary 3D shapes. The proposed whole system operated at 125 Hz, 6 times faster than the conventional system, and we verified the effectiveness of high-frequency feedback control and the proposed system through a series of simulation and real-world experiments - robot's catching and cooperative catching. Our system not only enhances the robot's ability to perform dynamic manipulation but also extends its operational range to accommodate dynamic environments.