Augmenting the bottom line with extended reality technologies

Many Australian small-to-medium manufacturers are yet to realise the benefits of low-cost extended reality (XR) technologies, said Anthony Franzè, Industrial Designer and Project Coordinator at the Advanced Robotics for Manufacturing (ARM) Hub.

Franzè is lead author of a new research publication on the use of technologies such as augmented reality (AR), mixed reality (MR), and virtual reality (VR) in manufacturing.

“Some innovative Australian manufacturers are taking advantage of XR technologies, to produce better quality, highly customised, low-volume, and bespoke products to service niche sectors of the global supply chain,” he said. “However, uptake remains low, despite the potential benefits across many facets of manufacturing, such as product design, prototyping, training, maintenance, fabrication and assembly.”

He said this was mostly due to the time and resource constraints manufacturers face, and unfamiliarity with how these Industry 4.0 technologies (advanced digitalisation) can be best applied in their processes.

The terms “AR” and “MR” are often used interchangeably. Both technologies enable synthetic overlays to be applied onto the real environment, while VR is a fully enclosed synthetic experience.

Franzè is employed at ARM Hub (while completing his PhD at QUT), which focuses on augmented fabrication and creating an Industry 4.0 XR framework for Australian custom manufacturers.

“My PhD explores how immersive technologies can improve Australian manufacturers’ design-to-fabrication processes,” he said. "The technology is very powerful in the sense that it overlays 3D virtual information at scale and directly on the context of what you’re working on, removing any task ambiguities.”

Franzè is the lead designer on an ARM Hub project with Young Guns Container Crew (YGCC), which used AR to simulate the design for a new container packing and unpacking tool that was demonstrated in digital simulations to potentially double productivity.

The project team developed a new ‘cartridge’ design, which allows a forklift to supply up to two empty pallets at a time, integrated rollers, a wrapping station, and an integrated blocker device. The team mocked up their distribution centre, containers, and the new cartridge tool in a 3D CAD (computer-aided design) model and transitioned the data into an augmented reality environment using Microsoft HoloLens 2 headsets.

The simulation used in the development of the tool tracks the actions and interaction of each aspect of container de-stuffing. Each unpacker, forklift, container and pallet are represented as an agent within a distribution centre.

Young Guns’ AR simulation of a new tool design for container packing and unpacking. The YGCC team were able to work within the synthetic overlay of images to simulate packing and unpacking a container using the new tool.

“We showed what the product would look like at various stages of utilisation in the distribution centre,” Franzè said. “We had YGCC’s frontline team come in, and they could put the headsets on, and they could walk around in the virtual future distribution centre environment with two people at a time.

“The team could discuss what it would be like to use the ‘cartridge’ device in different stages of operation, get a feel for it and develop a shared understanding of the device’s potentials. The AR was hugely beneficial for the team in terms of being able to engage and understand what the device is, because they didn’t just have to imagine being with it, they could actually be with it,” he said.

Franzè said these technologies helped reduce reliance on 2D drawings and physical templates and are also valuable for quality assurance tasks, and facilitate collaborative environments.

“Also with training, AR and VR are very helpful in training the workforce and upskilling existing staff,” he said. “I use it a lot to design and fabricate products, and to simulate and visualise. I find that AR and VR have a positive impact on the co-development process.”

Franzè said working at the ARM Hub was invaluable, enabling him to apply research learnings about the technology and development process to real-world situations.

He recently ran a design workshop at ARM Hub in collaboration with Australian AR/MR software development company Fologram, staff from QUT, the Australian Cobotics Centre and Brisbane-based bespoke public art manufacturers UAP.

“A group of 10 of us got together and fabricated quite a complex structure with all of us using Microsoft HoloLens 2 mixed reality headsets,” he said.

“There were 10 of us fabricating at the same time, and we had never used that sort of fabrication process before, but within an hour, the 10 of us, collaborating as a sort of hive mind, built this amazing structure. It was a real eye-opener on the power of technology, and we got quite a good result.”

Franzè says he sees immersive technologies as a tool to synergise human–robot and AI collaboration to assist manufacturers across tasks.

“I think immersive technologies will become a ubiquitous part of our society, of our lives — much like smartphones are now,” he said. “I think it will enhance our capabilities and extend the human experience beyond what is currently possible.”

Franzè is the lead author of a new research paper, Employing AR/MR Mockups to Imagine Future Custom Manufacturing Practices, co-authored by QUT colleagues Associate Professor Glenda Amayo Caldwell, Dr Müge Teixeira, and Associate Professor Markus Rittenbruch.

Top image caption: Anthony Franzè using an MR headset at ARM Hub.