International robot federated learning project a success

The FLAIROP (Federated Learning for Robot Picking) research project, which was sponsored by the German Federal Ministry for Economic Affairs and Energy and the Canadian National Research Council, was concluded at Festo in Esslingen-Berkheim in August.

Over the lifetime of the project, Canadian project partners focused on object recognition through deep learning, explainable AI and optimisation, while the German partners contributed their expertise in robotics, autonomous grasping through deep learning, and data security.

Over the past two years, Festo has been conducting joint research with the Karlsruher Institut für Technologie (KIT) and partners from Canada (University of Waterloo, Darwin AI) to make picking robots more intelligent using distributed AI methods. To do this, the partners investigated how robots can learn from each other without sharing their training data. This approach — called federated learning — allows the development of more robust and efficient AI than would be possible with data from just one robot, without handing out sensitive company data. The aim was to allow multiple robots — not only within a single site, but across multiple sites and organisations — to share their learning without compromising private company information.

If robots learn alone, the learning process is slow; however, if different robots doing unique picking tasks at different sites can pool what they learn, improvement will be much faster at all sites. The aim therefore was to do this while keeping data about bin objects, grasping speed and more, completely private.

The robot arms in the picking cells are equipped with cameras to visually detect the items in front of them. Based on the camera image, the robot arms automatically recognise the different items and select a suitable gripping method. Due to the variety of items in a warehouse, this is a complicated task and large amounts of data are needed to achieve reasonable results. Creating such large amounts of data is time consuming, so with data collected from picking cells in different organisations, it was possible to improve the grasping point detection of the cells.

In the FLAIROP project, a central AI system broadcasts a global training model to the bin-picking cells at different sites. Then, at each site, the model is trained and improved. After a period of time, the improved model from each site is sent back to the central AI, where it is aggregated and collectively updated. A new and improved version of the global training model is then sent back to all sites to start the next iteration. The process continues, to the benefit of each site, improving learning speed.

“We are proud that we have succeeded in showing that robots can learn from each other without sharing sensitive data and company secrets,” said Jan Seyler, Head of Advanced Development Analytics and Control at Festo. “This protects our customers’ data and we also gain speed because the robots can take over many tasks faster this way.”

“We have developed a universal, simulation-based data set that we can use to train autonomous gripping robots in such a way that they are able to reliably grasp items that they have not seen before,” explained Maximilian Gilles from KIT. “In the future, the federated learning system will be further developed so that the platform enables different companies to train robot systems together without having to share data among themselves. This can increase the acceptance of such systems in practice.”

During the project, a total of five autonomous picking stations were set up for training the robots: two at the KIT Institute for Materials Handling and Logistics Systems (IFL) and three at Festo SE & Co. KG based in Esslingen am Neckar.