Collective Intelligence Framework Can Fast Track Inspection and Accuracy

Technology #34286

Questions about this technology? Ask a Technology Manager

Download Printable PDF

Image Gallery
Example depiction of a bridge inspection using an AI-powered mixed reality system.
Enes Karaaslan, Ph.D.
Ulas Bagci, Ph.D.
External Link (
F. Nacati Catbas, Ph.D.
External Link (
Patent Protection

US Patent Pending
Artificial Intelligence Assisted Infrastructure Assessment Using Mixed Reality Systems
Transportation Research Record, July 17, 2019
Mixed Reality Assisted Infrastructure Assessment
Conference Presentation, SMT and NDT-CE 2018, New Brunswick, NJ
Enhanced Infrastructure Assessment Using Artificial Intelligence and Mixed Reality
The Smart Conference, Orlando FL, 2019
Enhanced Infrastructure Assessment Using Artificial Intelligence and Mixed Reality
Transportation Research Board 98th Annual Meeting, Washington DC, 2019

Key Points

  • Human-centered technique uses artificial intelligence (AI) and mixed reality to enhance inspections of civil infrastructures
  • Saves inspection time and lowers labor cost
  • Enables inspectors to perform a more objective assessment


Researchers at the University of Central Florida have developed a method for reducing the time and cost of inspecting civil structures like bridges and buildings. Conventional methods for visually assessing civil infrastructures use subjectivity and may require long inspection time and high labor costs. Although some technologies (such as robotic techniques, augmented reality, and mixed reality interfaces) can collect objective, quantified data, an inspector's expertise is still critical in many instances. Such technologies, however, are not designed to work interactively with a human inspector.

In contrast, the UCF Collective Intelligence Framework uniquely blends AI with mixed reality (MR) and can be integrated into an MR-supported see-through headset or a hand-held device. An inspector can analyze damage in real time and calculate or assess its condition without performing any manual measurements. At any step of the analysis/assessment, the inspector can intervene and correct the operations of the AI. Another advantage of the system is that the inspector can analyze defects in a remote location safely while reducing the need for access equipment. For example, if the defect location is hard to reach and poses a safety concern (like under a bridge or atop high-rise buildings), the headset can zoom in and still perform an assessment without needing equipment like a ladder. Consequently, the methods can reduce inspection time and labor costs while ensuring a quantified, reliable, and objective infrastructure evaluation with human-verified results. The methodology is expandable for many types of structures.

Technical Details

The UCF invention integrates unique AI detection and segmentation algorithms into an MR framework that performs automatic detection and segmentation of defect regions using real-time deep learning operations. In infrastructure assessment, creating a large image dataset for machine learning is essential but can be challenging. As a solution, the invention offers an advanced data augmentation technique that enables the framework to generate a synthetically sufficient number of images (like cracks and spalls) from available image data.

In an example application, an inspector assesses a concrete pier using an MR headset integrated with the framework. While the inspector performs routine tasks, the AI system in the headset continuously guides the inspector and shows possible defects in real time. Once the inspector confirms a defect location, the AI system starts analyzing it by first executing defect segmentation, then characterization to determine the specific type of defect. If the defect boundaries need corrections or the segmentation requires fine-tuning, the inspector can make the necessary adjustments. The system uses the alterations made by the inspector to retrain the AI model so that the AI's accuracy improves over time.

Partnering Opportunity

The research team is looking for partners to develop the technology further for commercialization.

Stage of Development

Prototype available.


  • Ensures reliable and objective infrastructure evaluation
  • Reduces the need for access equipment by enabling inspectors to zoom in on far locations
  • Allows effective use of data in infrastructure management systems
  • AI continues learning and improves its accuracy over time

Market Application

  • Transportation agencies
  • Building and construction inspection