Using AI and an app to identify components

AI methods have long been successfully used in image processing with great success. Neural networks recognise everyday objects with greater accuracy than humans. Research teams at the Fraunhofer Institute for Production Systems and Design Technology IPK employ these capabilities and adapt the algorithms for use in industrial applications.

Researchers have developed an app that enables individual components with no barcode to be unambiguously identified within seconds. This application of neural networks will particularly benefit logistics companies, which can use them to speed up their incoming goods processes.

Companies are increasingly producing goods at different locations, working with multiple supply companies. There is no guarantee that all the components they receive are labelled with barcodes or type plates, which means that objects often need to be recategorised in the receiving area. Employees then have to manually search the catalogue for similar parts to unambiguously identify them for further logistics processing — a painstaking and time-consuming task. Automated, digitalised recognition would be helpful to speed up the process. Researchers at Fraunhofer IPK in Berlin are applying machine learning methods with convolutional neural networks (CNNs) to recognise manufactured components, such as screws, clamps, nozzles, pipes, tubes and cables, as well as microcontrollers and other electronics.

Generating training data for industrial processes

“CNNs have become the standard in image processing. Reliably recognising, say, 1000 everyday objects requires one million images for these networks to use as training data,” said Jan Lehr, a researcher at Fraunhofer IPK. “Our job was to generate an algorithm for industrial applications even with little data so that, in our case, components with no code can be recognised automatically, easing the burden on receiving staff.

“Our goal is to enable the algorithm to easily distinguish even highly similar objects from each other, such as screws of the same standard but different sizes, or turbochargers from different production series.

“We use specially developed algorithms to limit the search radius to five or 10 objects, so employees no longer have to search through the entire range typically found in a large warehouse.”

To realise this, Lehr and his colleagues, working on a project funded by the German Federal Ministry of Economics and Energy (BMWi), developed a detection system called Logic.Cube. The objects to be recognised, which have a maximum edge length of 40 cm, are placed in the cube-shaped device with an integrated scale and are photographed by up to nine cameras. An image processing algorithm measures the objects’ height, width and length in order to calculate what size box or shelf space is needed. At the same time, the resulting image set is stored in a database, together with the material number. This image data is used to train the AI algorithm to enable it to recognise a wide array of different components.

High recognition rates with few images

Since not every company will consider purchasing the Logic.Cube, the Fraunhofer IPK research team ported the detection system’s functionality to a browser-based, operating system-independent app that works on smartphones, tablets, laptops and desktop computers. To do this, they had to expand the training dataset to include smartphone data and retrain the algorithm.

“We tested the algorithms with a hundred components that had been photographed in a wide variety of scenes, with 50 images taken of each part. Within seconds, the app shows the user five or fewer potential matches, independent of lighting, background and scenery. The recognition is so robust that it can replace manual searching,” said Lehr. “This saves workers an incredible amount of time. We managed to achieve high recognition rates with a minimum of images.”

The researchers reached recognition rates of 98% in Logic.Cube and reduced the search radius from 4500 images to five. They aim to get the same success rate with the app.

The images will be uploaded over the internet or the company’s intranet and stored in a local edge cloud, which is also where the actual image processing and recognition will take place.

“The AI algorithms run on the server, and the smartphone or tablet app is the client,” explained Lehr. The overall system is designed in such a way that, when used, it continuously collects additional data. After a certain amount of time, this new data can be used to retrain the algorithms, resulting in a continuously self-improving system.

The app interface. Image ©Fraunhofer IPK. For a larger image, click here.

Currently the researchers are working to expand the set of image data and to digitalise the catalogue and integrate it into the app. It should soon be possible to initiate all subsequent processes, such as further processing and potential reorders, through the edge cloud. Lehr and his colleagues are also optimising the algorithms to enable recognition of even highly similar-looking objects. In the test runs they conducted, the system correctly recognised even screws of the same standard but different sizes.

Application scenario: surface inspection

The app can also be used for surface inspection. AI-based image processing is used to examine the surfaces of incoming components as soon as they are received. It shows whether parts have any scratches or corrosion, or whether they were damaged during transport or are just a bit dirty. AI methods are used to mark the relevant spots on the image so workers can immediately check the damaged areas.

“Manufacturing companies are often still cautious when it comes to AI,” said Lehr. “We would be very pleased if our research work could help counter this scepticism and boost acceptance among employees.”

Top image: The app recognises a wide array of different components. Image ©Fraunhofer IPK.