Located near Naples, the once-thriving city of Pompeii and its surrounding countryside were submerged by volcanic ash when Mount Vesuvius exploded in A.D. 79. Ironically, the eruption preserved much of the city, even as it killed thousands of people.

Among the site’s most treasured items are the original frescos inside homes and businesses. Although some frescos are still on the walls, others are in a fragmentary state. Archaeologists want to reassemble the paintings and restore them to their former glory.

The process is not easy. “It’s like you buy four or five jigsaw puzzles. You mix everything together, then you throw away the boxes and try to solve four or five puzzles at the same time,” said Marcello Pelillo, Ph.D., a professor of computer science at the University of Venice and the coordinator of the robotic project.

The system consists of two robotic arms equipped with soft, flexible hands in two sizes. The hands can grip and assemble fragments without damaging their delicate surfaces. Vision systems identify specific fragments. 

Experts in artificial intelligence and machine learning developed algorithms to reconstruct the frescoes, matching colors and patterns that may not be visible to the human eye. Experts say the task is similar to solving a giant jigsaw puzzle, with extra difficulties such as missing pieces and no reference image of the final result.

After acquiring and digitizing images of the individual fragments, the system attempts to solve the puzzle, at times engaging with archaeologists who can identify errors or offer suggestions. If the system identifies a match, it directs the robots to position the fragments in the desired locations.