Not just Pokémon – prepare for an augmented reality deluge

Augmented reality took the public by storm this summer, driving 100 million gamers to search for Pokémon hidden around their cities. But while the monster-chasing craze may fade, it won't be the last you hear of the technology behind it as researchers look beyond the entertainment industry and see how it can be put to work to improve our daily lives.

The technology works by tapping into the cameras and GPS systems of smartphones to overlay computer-generated images onto real-world surroundings when viewed through a touchscreen. Being able to enhance people's views of their surroundings in this way is opening up myriad possibilities for design studios, archaeologists and even the construction industry.

Researchers in Italy want to provide museums, churches and cultural heritage sites with the means to guide visitors through each historical nugget hidden within their stones, by creating building models overlaid with information that people can access as they point their phones at different parts of a site.

'Curators have to tread a thin line between restoring monuments in a way that appeals to tourists and conserving their history for future generations,' said Professor Roberto Di Giulio at the University of Ferrara, Italy. 'Now augmented reality can display our historical heritage in levels of detail that were previously unimaginable, all without touching the original building.'

The EU-funded INCEPTION project that Prof. Di Giulio coordinates is searching for ways to automatically break down computerised building models into architectural features. Curators can then load the digital version of these features with interactive information that visitors can access through a tablet or a mobile phone and compare what they see around them.

They are using a technique known as 3D laser scanning. Its essence is to shine a laser in billions of directions inside each room and infer the distance to each point from the angle or the delay in recording the reflected beam. Project partners are currently refining the technique to make it accessible to cultural heritage sites all over the world.

'The technology offers a virtual time machine that reveals how buildings were used in the past and how they evolved over centuries,' said Prof. Di Giulio. INCEPTION is demonstrating its potential in eight test cases, including a 3D model of Filippo Brunelleschi's building Ospedale degli Innocenti in Florence that could take visitors on a journey back to the 15th century.

In the 21st century, augmented reality is being put to work on another aspect of buildings - how to construct them in the first place.

Dr Rizal Sebastian at DEMO Consultants in Delft, the Netherlands, said that he sees the need for this type of virtual help because his experience on building sites has taught him that even experts make mistakes.

'Modern buildings are designed using prefabricated components and sophisticated computer 3D models that tell you exactly how each piece fits together,' said Dr Sebastian. 'Still, buildings rarely meet their energy efficiency expectations, partly because human errors crop up during their construction.'

He coordinates the EU-funded INSITER project which began at the end of 2014 and aims to translate the designer's building model into step-by-step instructions that appear on mobile devices carried by construction workers. The software repeatedly compares the construction site with the 3D computer model and detects clashes between the two.

'We are trying to automatically spot assembly mistakes with field measurements using lasers, and thermal and acoustic probes,' said Dr Sebastian. 'The result could help builders check each step of the construction process along the way.'

Communication

Elsewhere, Professor Gaetano Cascini at the Politecnico di Milano in Italy sees an overlooked opportunity for augmented reality to help human beings communicate with each other. His research aims to smooth the collaboration between users and designers in creative sessions.

According to Prof. Cascini, companies are increasingly recognising the value of feedback from the public in designing new gadgets and appliances. But brainstorming with non-specialists runs up against obstacles. Consumers often struggle to visualise abstract ideas from blueprints and slide shows.

In the EU-funded SPARK project, Prof. Cascini leads a consortium that uses light projectors to beam colour onto life-sized plaster casts of new products. Consumers can handle the casts to get a better feel for the final product and designers can alter their appearance in real time.

One challenge is for the light beam to follow the cast around the room without altering the image projected on its surface. To do so, an algorithm works out the position and orientation of each cast from beacons fixed to its extremities and adapts the image sent through the projector accordingly.

The highlight of the SPARK approach is that it manages to augment reality without using any screens. 'This frees designers up so that they can focus on being creative,' said Prof. Cascini.

He is now working on gesture recognition techniques that automatically interpret the hand movements that emerge most frequently in the design room. Already consumers can activate buttons that are projected on the plaster casts and designers can change their fonts and patterns with a wave of the hand.