INR-DOM (Implicit Neural-Representation for Deformable Object Manipulation) enables robots to skillfully handle objects with shapes that continuously change and are visually difficult to distinguish. The AI-powered system allows machines to completely reconstruct the overall shape of a deformable object from partially observed three-dimensional information and to learn manipulation strategies based on it.

In addition, a two-stage framework combines reinforcement learning and contrastive learning so that robots can efficiently learn specific tasks, such as how to untie entangled rubber bands.

“Deformable object manipulation is one of the long-standing challenges in robotics,” says Daehyung Park, Ph.D., an associate professor computer science and head of KAIST’s Robust Intelligence and Robotics Lab. “This is because deformable objects have infinite degrees of freedom, making their movements difficult to predict. And, the phenomenon of self-occlusion, in which an object hides parts of itself, makes it difficult for robots to grasp their overall state.

“To solve these problems, representation methods of deformable object states and control technologies based on reinforcement learning have been widely studied,” explains Park. “However, existing representation methods can not accurately represent continuously deforming surfaces or complex three-dimensional structures of deformable objects. And, since state representation and reinforcement learning are separated, there is a limitation in constructing a suitable state representation space needed for object manipulation.”

To overcome these limitations, Park and his colleagues utilized implicit neural representation. This technology receives partial three-dimensional information observed by a robot and reconstructs the overall shape of the object, including unseen parts, as a continuous surface. This enables robots to imagine and understand the overall shape of the object, just like humans.

When the INR-DOM technology was mounted on a robot and tested, it showed overwhelmingly higher success rates than the best existing technologies in three complex tasks: inserting a rubber ring into a groove, installing an O-ring onto a part and untying tangled rubber bands.