Researchers Combine Mobile 3D Handheld Scanner With Robot Dog

With the handheld scanner goSCOUT3D, Fraunhofer IOF researchers have developed a mobile sensor that enables high-resolution 3D measurements of complex objects. To further automate this process, researchers have now connected the scanner to a robot dog of the Boston Dynamics brand. Doing so is supposed to make measurements, e.g., in quality assurance, even more efficient as well as unburden employees in future. The unit consisting of sensor head and robot will now be presented for the first time at the CONTROL trade fair in Stuttgart from April 23-26.

“Heel”, “sit”, or “down” are commands every dog owner is familiar with. In production halls or industrial manufacturing plants, on the other hand, they are rather less likely to be heard. But if you ask researchers at the Fraunhofer Institute for Applied Optics and Precision Engineering IOF in Jena, perhaps in future you may also encounter a four-legged companion here. This helper works according to strict human instruction and thereby makes the day-to-day work easier for its two-legged colleagues. Except that the dog in this case is not a living animal, and the commands primarily apply to tasks for high-precision 3D documentation.

Automated 3D digitization with robot dog Spot

Be it for the servicing of complex components such as aircraft engines, or for quality control in the automotive industry: Virtual 3D models enable the precise documentation of large machines and their small details. For this purpose, researchers at Fraunhofer IOF have already developed the mobile handheld scanner goSCOUT3D in the past year. It enables 2D and 3D digitization of complex objects with a volume of several cubic meters, at a particularly high spatial resolution of less than 0.25 millimeters.

Until now, to create these so-called “digital twins”, the scanner had to be moved manually by human operators around the object to be measured. Operators were required to carry out the measurement with great precision, to guide the sensor head at a constant speed and to maintain a largely constant measuring distance. Potential signs of fatigue or incomplete measurement data were the result at times.

Researchers from the department of Imaging and Sensing at Fraunhofer IOF have addressed this problem from the shop floor and searched for a way to make the measurements more comfortable for humans, and at the same time make the operational process more efficient. The solution: They combined the 3D scanner with a robot dog named Spot from the US company Boston Dynamics.

“Integrating the sensor head, which was recently developed in Jena, with Spot, will automate the measuring process of the goSCOUT3D scanner in future and eliminate the need for constant human supervision”, explains Dr. Andreas Breitbarth, head of Image Processing and Artificial Intelligence at Fraunhofer IOF. “The robot dog can autonomously manoeuvre goSCOUT3D through measuring scenarios at consistent measuring speed and distance”.

Robot makes measuring process even more agile and flexible

Compared to conventional measuring robots that are, e.g., employed along production lines and ordinarily permanently mounted in place, the autonomous walking robot has a crucial advantage: Thanks to its ability to move around freely and its agility on diverse surfaces, the measurement object can be recorded in full, from every side as well as on various levels, such as below a guide rail.

Consistent scan speed and reduced handling variability furthermore ensure greater reproducibility of results and decrease the measurement time through fewer redundancies. That makes the integrated measuring unit particularly interesting for application in serial manufacturing processes or quality control.

Autonomous measuring without human interaction

To achieve these application goals, a reliable coordination between the robot and 3D sensor is necessary. So, a hand-eye-calibration is used to connect the 3D sensor to the robot’s coordinate system. As soon as the walking robot is calibrated to the scanner in this way, it can be programmed by human experts to measure predefined objects. “Before the actual measuring process, speed, precise measuring location, and other important parameters are submitted by a human operator to Spot, so that the robot dog can conduct the scans the same way as a human being, without the need for them to be physically present”, Breitbarth explains further.

Thus, Spot could take on, e.g., routine measurements in future. The human experts could instead focus on critical aspects of review as well as analysis and evaluation of the 2D and 3D data created with the goSCOUT3D. Furthermore, linking the scanner to an agile robot is also expected to enable methods for remote operation and monitoring.

goSCOUT3D enables mobile measurement of complex 3D objects

Researchers at Fraunhofer IOF developed the 3D handheld scanner goSCOUT3D in 2023 in collaboration with MTU Maintenance. To create complex 3D models, the sensor head, which weighs only 1.3 kilograms, captures two-dimensional color images of the selected measurement scenario using a 20-megapixel color camera. The 3D data of the entire scenario is calculated from these using photogrammetry.

Presentation at CONTROL 2024

The unit, consisting of the goSCOUT3D and Spot, will now be showcased at CONTROL 2024 for the first time. At the international trade fair for technology and quality assurance in Stuttgart, the measuring system will be presented from April 23 to 26. Here, Fraunhofer IOF will exhibit at booth 8201 in hall 8.