Robust Robotic Grasping via Teacher-Student Learning and Informed Point Cloud Sampling

Abstract

Current sim-to-real methods process sensory data uniformly, leading to computational inefficiency and problems with the sim-to-real transfer, as policies tend to overfit to scenes, rather than learn robust features. Drawing inspiration from the human selective gaze mechanism, we present a novel method called informed point cloud sampling to address these issues in reinforcement learning with point clouds. Our method can be applied within a Teacher-Student framework to prioritize task-relevant regions. By incorporating an auxiliary distance estimation head during training, our system can effectively identify object centers through the combination of distance estimates and current end-effector positions. This can be further exploited to generate object-centric observations, removing irrelevant information and increasing robustness to different settings. We apply our proposed method to robotic grasping in the real world. Experimental results demonstrate that our method achieves performance comparable to baseline methods while using significantly reduced point cloud density, improving computational efficiency, and leading to a robust sim-to-real transfer. Our method’s effectiveness is validated through comprehensive simulation and real-world experiments, showing promise for robust robotic grasping.

Method

Overview of the Teacher-Student training process. First, we train a teacher policy on privileged information using Reinforcement Learning. After the policy achieves a sufficient success rate, we use the teacher in an imitation learning process to generate target actions, that the student imitates. Additionally, the student estimates the current distance between the grasp position and the object center. We use this estimation for the informed sampling process, which generates an object-centric point cloud, which we merge with a synthetic point cloud representing the robot.

Uniform Sampling (left) vs. Object Tracking with Auxiliary Head and Informed Sampling (right)

Here as a visuell example we show the common way of uniformely sampling and our informed sampling-method from a point cloud. We only use the object estimation output generated by the policy to set the object center and sample the point cloud using our proposed method. The policy is able to track the object on the table.

Quantitative Experiments

To evaluate the proposed methods, we perform an ablation study to assess the extension to the baseline in simulation. Further, we evaluate the efficiency of the proposed point cloud sampling method. In real-world experiments, we investigate the trained policies in terms of grasping success and robustness to deviations, such as different scenes, perturbations, and camera positions.

Grasping Experiments

The grasping experiments we conducted of all twelve objects.
\[ \begin{array}{l | c c | c } \textbf{Object} & \textbf{Ours} & \textbf{Avg. Grasp Time} & \textbf{Wang et al.} \\ \hline Screwdriver & \textbf{5/5} & 9.00\,s & N/A\\ Can & \textbf{4/5} & 9.75\,s & 3/5\\ Mug & \textbf{5/5} & 8.20\,s & 4/5\\ Banana & 5/5 & 14.20\,s & N/A \\ Brick & 5/5 & 9.60\,s & 5/5\\ Soup Can & 3/5 & 15.70\,s & 3/5\\ Sugar Box & \textbf{5/5} & 8.60\,s & 4/5\\ Cracker Box & 2/5 & 17.50\,s & \textbf{3/5}\\ Mustard & 4/5 & 12.50\,s & 4/5\\ Ball & 4/5 & 22.25\,s & N/A\\ Bowl & \textbf{5/5} & 9.80\,s & 4/5\\ Bleacher & 4/5 & 13.50\,s & 4/5\\ \hline In Comparison & \textbf{37/45} & - & 34/45\\ Success Rate & \textbf{82.2%} & - & 75.6\,\%\\ \hline All & 51/60 & 12.54\,s & -\\ Success Rate & 85.0% & - & - \\ \end{array} \]

Qualitative Experiments

Invariance to Changes in the Scene

In this experiment we perform two grasps of a mug, then change the camera and move the surface, from which we grasp the object. Then we perform two more grasp and change the camera and desk again, to perform two final grasps. This shows, that our method works without any strong synchronization of simulated scene and reality.

Different Scene and Camera Angle

We turn the desk, that the robot is situated on towards another surface, that is strongly out-of-distribution from the environment, that we trained the policy in. Furthermore, the camera angle and the robot pose is also not included in simulation training. However, by leveraging informed point cloud sampling and the sim-to-real methods we propose, the policy still has some success in grasping objects.

Grasping with Distractors

We test the method on performing in scenes with different objects. If there is a big difference between the object to be grasped and the distractors, the policy can easily choose to grasp the right object thanks to informed sampling.

Failures

Here, we depict some failures of the system. To be precise, grasps where the system took especially long to perform the task or where a human had to intervene. The most common failure case is slightly failing in the grasp due to precision and then repeating this behavior over and over again. Sometimes the system also fails in estimating the state correctly resulting in an orientation that doesn't allow a successful grasp.