When designing vision inspection systems for the construction industry, you have some unique challenges to face:

1. Your systems work in difficult environments, often dirty, rough and hard to access
2. You have only a low number of actual defect images per category to train models
3. You face quickly changing requirements and camera positions

In the following article, we would like to share some of our experience with vision inspection systems in the construction industry. You have different experience or think we missed something relevant out?

We are looking forward to receiving your feedback via contact@preml.io

How do vision inspection systems work?

With the invention of computers, traditional vision inspection, in which algorithms analyze images to inspect components, entered the stage. There are plenty of use cases for such systems, from automated inspection, to guidance of robots, quality control, sorting tasks and many more. In its simplest form, a vision system consists of at least one camera and a computer to run the software. Vision systems can measure parts, verify their correctness in position and geometry and recognize shape and color. The system can be integrated in current processes and provide constant information and results. It also can proceed this information and make decisions, such as a pass/fail decision which triggers an operator or robot to act.

At preML we always ask one simple question as a first prerequisite, to ensure a vision inspection system is capable of solving the problem: “Given the image can a person solve the recognition task with their eye?”. If that’s the case, many such as the aforementioned problems can already be solved by computer vision systems.

Beside the technical feasibility, obviously, situations in which the system leads to a profit in form of safety, quality or in profit are beneficial. Most obvious use cases are everywhere where parts for construction are produced in line, or circulation, such as prefabricated parts. The same applies to the installation of such parts in a machine-like process. More difficult, but worthy under certain conditions are maintenance tasks, such as the observation of constructions which are difficult to access (e.g. bridges, dams, wind turbines).

What are the challenges for vision inspection in construction?

We mainly face three challenges at our projects:

1. The systems work in difficult environments, often dirty, rough and hard to access

This point does not need a lot of explanation. In most projects we have to deal with dust, water, or mud and cameras might be installed in very high positions, or very narrow ones or even underground or on flying objects.

2. There is only a low number of actual defect images per category to train models

For decades, the industry has focused on avoiding errors through process optimization. For computer vision, especially if Machine Learning (ML) is used, which learns strong models from large data sets, this is a challenge. Many of our customers lack a database of defective parts. Some may have 100, 10 or even fewer images of a defect they would like to detect.

3. There are quickly changing requirements and camera positions

With the constant improvement of products and processes, there is an increasing chance that vision inspection systems will quickly fail to meet decided and trained conditions. Product changes and changes in the production environment put inspection systems at risk and can even lead to the discontinuation of the original system.

How to design successful systems?

In the following we share some key learnings in designing and implementing a successful system to overcome the mentioned challenges.

a) Autonomous working and specially protected systems

Even though it is technically possible to use cloud computing for most of our applications we decided in all of our current projects to use edge devices. This allows us to avoid networking issues and guarantee a maximum reliability. Obviously, we also found ways to protect our hardware equipment according to the unique environment and fulfill the often very strict safety requirements (e.g. ATEX).

b) Be flexible in your model creation

Having few images of defects are actually one of the most complex challenge the full machine vision industry and research face. Probably we will discuss this issue one day in a special blog post. Anyway, there are some things we do to overcome such issues:

• Combine traditional methods with ML – many times traditional algorithms are more reliable and also cheaper to develop
• ML models which trains with “only good” data. We successfully refined unsupervised and semi-supervised algorithms for anomaly detection to target domains inside the construction industry. (this one is kind of a recipe secret of our company, which is why I won’t give too many details at the moment)
• Use cross-project datasets for model training
• If the upper points don’t work, go the extra mile to collect high quality data. Once we literally traveled through Europe to create a dataset.

c) Be flexible in error definitions and expect changes in environment

In production, conditions are adjusted. An accepted tolerance for a crack width of 0.4 mm may be 0.2 mm the next month. Further, the definition of a defect can be very subjective and it is common for disagreements to arise between two inspectors. Therefore, our annotation and software development processes in cooperate and expect changes in environment and specifications.

Summary

Vision inspection systems can dramatically simplify and improve quality processes in the construction industry. Herein, ML is not the solution to all problems, often traditional methods are still the best option to solve a problem. However, there are various uses of ML to make processes more effective, autonomous and cheaper. As in any project, it pays out to define the specification clearly and to research and weigh problem solutions well. By asking the right questions and using the right tools in production, it is possible to create value with ML in even harsh environments such as the construction industry and make your processes more efficient and better.

About us

preML GmbH is a computer vision start up that develops customer-specific solutions in the field of image processing and object detection for industry and the construction sector. One focus is on the detection and monitoring of building elements, especially those made of concrete, with the help of camera technology and intelligent algorithms. The start-up was founded in 2020 at KIT from the Department of Computer Science, and the company’s current headquarters are in Lahr in the Black Forest.