Robotic Autonomous Trolley Collection

Description

(Trolley collection robots working at an airport^[1]. )

This work presents a novel mobile robotic manipulation system with applications in luggage trolley collection. The proposed system integrates a compact hardware design and a progressive perception and planning framework, enabling the system to efficiently and robustly collect trolleys in dynamic and complex environments.
To the perception end, we develop a 3D trolley detection method that combines object detection and keypoint estimation. A docking process at a short distance featuring an accurate point-cloud-based plane detection method is also developed.
To the planning end, we formulate the robot’s real-time motion planning under a nonlinear model predictive control (NMPC) framework with control barrier functions (CBF). In this way, we are able to improve the obstacle avoidance capability of the robot while taking active observations (maintaining the target in the sensors’ field of view) into account when planning at close distances.
We demonstrate our design and framework by deploying the system on actual trolley collection tasks, involving both static obstacles and moving humans. The effectiveness and robustness of the system are experimentally validated.

Our contributions are four-fold. Concretely, we

develop a novel robotic autonomous trolley collection system integrating a mechanical system and an efficient autonomy framework;
propose a progressive perception strategy involving long-distance keypoints-based monocular 3D detection and short-distance accurate pose estimation using LiDARs;
present a safety-critical motion planner formulation under an NMPC framework with CBFs considering obstacle avoidance and observation constraints;
demonstrate the system in complex and dynamic environments detecting target trolleys and safely collecting the trolleys.

See below our ICRA 2022 presentation video introducting this work.
It includes a demo of deploying our system in a complex and dynamic environment to accomplish an actual trolley collection task.

(YouTube link here)

(bilibili link here for friends inside the GFW)

Global decision-making
- Perception-aware planning
Multi-robot collaborative manipulation.
Safe human-robot interaction:
- human model building, tracking, and prediction
- considering the interactive nature of autonomous agents (humans & other robots)
- safety-critical planning under uncertainty
Field deployment at real-world international airports.

* indicates co-first authors.

A. Xiao*, H. Luan*, Z. Zhao*, Y. Hong, J. Zhao, J. Wang, and M. Q.-H. Meng. “Robotic autonomous trolley collection with progressive perception and nonlinear model predictive control.” 2022 International Conference on Robotics and Automation (ICRA). Philadelphia, PA, USA, May 23-27, 2022, pp. 4480-4486.
- Preprint on arXiv: https://arxiv.org/abs/2110.06648.