Our built environment is aging and failing faster than we can maintain it. Recent building collapses and structural failures of roads and bridges are indicators of a problem that’s likely to get worse, according to experts, because it’s just not possible to inspect every crack, creak and crumble to parse dangerous signs of failure from normal wear and tear. In hopes of playing catch-up, researchers in Drexel University’s College of Engineering are trying to give robotic assistants the tools to help inspectors with the job.

Augmenting visual inspection technologies — that have offered partial solutions to speed damage assessment in recent years — with a new machine learning approach, the researchers have created a system that they believe could enable efficient identification and inspection of problem areas by autonomous robots. Reported in the journal Automation in Construction, their multi-scale system combines computer vision with a deep-learning algorithm to pinpoint problem areas of cracking before directing a series of laser scans of the regions to create a “digital twin” computer model that can be used to assess and monitor the damage.

The system represents a strategy that would significantly reduce the overall inspection workload and enable the focused consideration and care needed to prevent structural failures.

....

Instead of a physical measurement interpreted subjectively by human eyes, the system uses a high-resolution stereo-depth camera feed of the structure into a deep-learning program called a convolutional neural network. These programs, which are being used for facial recognition, drug development and deepfake detection, are gaining attention for their ability to spot the finest of patterns and discrepancies in massive volumes of data.

Training the algorithms on datasets of concrete structure images turns them into crack crack-spotters.1

Once the “region of interest” — the cracked or damaged area — is identified, the program directs a robotic arm to scan over it with a laser line scanner, which creates a three-dimensional image of the damaged area. At the same time a LiDAR (Light Detection and Ranging) camera scans the structure surrounding the crack. Stitching both plots together creates a digital model of the area that shows the width and dimensions of the crack and allows tracking changes in between inspections.2

The team tested the system in the lab on a concrete slab with a variety of cracks and deterioration. In a test of its ability to detect and measure small cracks, the system was sensitive enough to pinpoint and accurately size up the smallest of fissures — less than a hundredth of a millimeter wide — outperforming top-of-the-line cameras, scanners and fiber optic sensors by a respectable margin.

---

• 1. “The neural network has been trained on a dataset of sample cracks, and it can identify crack-like patterns in the images that the robotic system collects from the surface of a concrete structure. We call regions containing such patterns, regions of interest,” said Ebrahimkhanlou, who leads research on robotic and artificial-intelligence based assessment of infrastructure, mechanical and aerospace structures in Drexel’s Department of Civil, Architectural, and Environmental Engineering.
• 2. “Tracking crack growth is one of the advantages of producing a digital twin model,” Alamdari said. “In addition, it allows bridge owners to have a better understanding of the condition of their bridge, and plan maintenance and repair.”