How Can Augmented Reality Transform The Manufacturing Floor?

  • Tuesday, 29 August 2017 09:37

APMEN speaks to Dr Antonio Feraco, chief business development officer at the Fraunhofer IDM@NTU project centre, on how augmented reality can have help manufacturers with processes such as remote maintenance, repair and overhaul. All images contributed by Fraunhofer IDM@NTU.

Various applications for augmented reality (AR) have already made their way to the manufacturing floor. For example, modern manufacturing involves putting together a great number of components in sequence as quickly and precisely as possible. Work instructions therefore tend to be static documents that are dozens of pages long.

Augmented reality software takes those instructions and places them into the field-of-view of the operator’s augmented reality device (such as glasses, tablet or projector) with voice control, giving prompts on what should be assembled next. This early indicators can avoid potentially costly corrections further down the line.

Such features are already being used in aerospace engineering. Airbus, for example, developed the MiRA (Mixed Reality Application), which scans parts and detects errors.

The MiRA reduced the time needed to check tens of thousands of brackets in the fuselage from 300 hours to just 60 hours. Furthermore, late discoveries of damaged, wrongly positioned or missing brackets have been reduced by 40 percent.

We speak to Dr Feraco to find out what other real-world benefits that augmented reality can bring to manufacturing.

Q: Seeing as the Fraunhofer IDM@NTU Project Centre focusses on digital media, how did you come into the manufacturing sector?

Detecting machine wear can be as easy as pointing a
tablet at it.

Dr Antonio Feraco (AF):We came into manufacturing in 2011 because that was when Industry 4.0 started, which is basically a new revolution in information and data exchange.

One of the main frameworks of Industry 4.0 is cyber physical systems. Some people term it as a virtual mirror, or a convergence of data.

From our perspective, Industry 4.0 is basically focussing on four main pillars. Before that, everything was on the product. Now you have to consider different aspects that are related to the product.

These aspects are used to optimise the product itself. These four pillars are organisation, business environment, people, and technology. So you can see that with the use of IT infrastructure in the manufacturing plants/industry, you also need different skills for workers.

Q: To go into more detail, what different types of skills are needed?

AF:Today’s workers need to have an interdisciplinary curriculum that ranges from engineering, to IT skills, and also the skills that are related specifically related to the product that they are manufacturing.

Universities are now changing their disciplinary paths. Here at NTU, one of the leading universities in the world, they already have interdisciplinary paths. Under the School of Computer Science and Engineering, the university also has subjects such as graphic design, 3D modelling, and rendering that are also part of the curriculum.

We are a subsidiary of NTU, and the university is one of our major partners here in Singapore. The university has been very supportive towards us. It is not only a question of brand, but a question of facts and the value-add to the local market and training of the local people here as well as the students.

Q: Currently, how is AR applied in manufacturing work processes (and IN which sectors)?

AF:There are some pilot programmes around the world with MNCs and in Asia that are adopting AR. The value can be seen in the optimisation of the work processes.

Within the work processes, we also consider the training of the workforce. Consider for example, reading manuals and books to learn how the processes are done, or being in the field using tablets and seeing the information on the spot; that is also how AR can bring value to the industry.

Augmented reality is basically the use of devices where you can add layers on top of what you see. So this can also be applied to quality control. Products can then be benchmarked with this approach.

Apart from this, you can also consider AR as a tool to visualise the environment data that are produced by the sensor devices included in the machinery, for real-time monitoring of the machines. Just pointing the tablets to the machine will help you recognise which machine or which position you are pointing the tablet at; the tablet will tell you the machine’s behaviour.

This is done via the machinery constantly sending data to the server. With the AR software that we developed here, if we point to a machine, the tablet will contact the server and send real-time information to the tablet.

The person who is in charge of the maintenance can have this real-time information wherever he is; what the machine is performing, if there is some abnormal behaviour of data detected by the sensors, and so on.

This provides a different reaction time in case something is not going well. In this application, predictive analytics can be applied here so you know when maintenance is needed, or if the behaviour is abnormal because of the maintenance.

Q: Other than highly specialised work, how can AR be used in industrial and manufacturing applications?

AF: A practical example is as follows: The operator’s application detects that it is looking at a workpiece, and visualises its data and embedded additional files (such as the manual). So you can interact with the piece of equipment accordingly.

So you can come up with the 3D wirework of each individual parts and other information on the spot. This is where I see the value-add of AR, you don’t have to dig up the paper manual or plans; the information is all in the application.

So you can have the device attached to an arm for hands-free handling; sensors can also be already embedded within the workpiece so you can get immediate and real-time feedback for this.

Q: Can it enable or enhance productivity and/or ROI in manufacturing? How?

Heat chart of a Rolls Royce engine.

AF:My own perspective on this is that it is not so straightforward to quantify the ROI when it comes to innovative technologies. But from a manpower perspective, this can speed up the training process of the workforce by training them on the field, for example.

Most of the time when you want to train someone on the production line, you also face some risks. On the manufacturing floor, there risks involved. If you do not apply the proper force or procedure, you can damage the machine itself.

With the use of augmented reality, you can have a better understanding on how to use the device, as well as learning the overall procedures to be done.

So with the example above, you can see things differently and have a different perception of things. Also with VR and the use of 3D glasses, you can be embedded in the virtual world where you can see how machinery performs, where it is in the production line, and so on.

You can also mirror the behaviour of the machine in real-time. Imagine you have a plant in Indonesia, but you are based in Singapore—without using data and excel sheets, you can see how the machines are working in Indonesia from Singapore via AR.

If you want to train your people, you can embed them in the virtual world that mirrors the reality of the plant they will be working at. So when you send them to the actual workspace, instead of taking time to apply what they have learnt to the physical environment they’ve never seen before, they will have a faster uptake as they have been training with the virtual reality-mirrored environment.

Q: So let us say if there is an issue with the plant in Indonesia. Can you fix the problem in Singapore using AR?

AF: Yes. In Germany, we have been working on remote Maintenance, Repair and Overhaul (MRO). You do not have to take 12 hours to send a technician to fly in from Germany to Indonesia.

To put it simply, you can use the tablet to point at the machine that is having the issue. The information and video feeds will be made available to the remote technician based in Germany.

So he will then provide instructions and recommendations based on what he sees and observes from the data to the Indonesia-based technician on the field in real-time.

So MRO is very prevalent in the aerospace sector, and some companies are also approaching the use of AR in these ways. This is because if they have operations around the world, they understand that building such solutions is saving costs instead of having to send specialised technicians and engineers all over the world. You also save time, and I think time and cost are directly related.

Q: Companies based in Asia are usually SMEs. How can adopting such practices enhance value?

AF: It is not only about training the workforce, but also trying to reshape the procedures. For example, with quality control, we can use the cameras to view the products. and scan it to receive information on tolerances, which then can be benchmarked on the spot.

About The Fraunhofer IDM@NTU Project Centre

Fraunhofer Society is an application-oriented research organisation. They established a presence in Singapore in 1998 with a memorandum of understanding with NTU. In 2010, they opened a technology centre in NTU called the Fraunhofer IDM@NTU.

The “IDM” represents “interactive digital media”, and the centre’s focus is specifically on technologies related to science and engineering. From 2016, the centre started their second phase under the RIE2020, a program under the National Research Foundation.

According to Dr Feraco, the centre’s approach relies on applied research, and comprises nine levels where level one is the idea of the product, and level nine is the final product. He added that their AR work is at around level seven, where it is being tested in real-life scenarios.

APMEN Feature, August 2017

Rate this item
(0 votes)
  • Last modified on Tuesday, 29 August 2017 09:56
  • font size




As Asia's number one English metalworking magazine, Asia Pacific Metalworking Equipment News (APMEN) is a must-read for professionals in the automotive, aerospace, die & mould, oil & gas, electrical & electronics and medical engineering industries.

Connect with us: