Once a technology of the future, virtual reality is now being used in everything from computer gaming to archaeology and medical research. And now VR is being joined by a related technology known as augmented reality (AR), which lets users view the real world overlaid with computer-generated imagery and information.
Many people are getting their first look at the technology via smartphone applications. Street View on Google Maps, for example, lets users explore a remote area by moving their phone around. AR is also used in museum installations, such as the Skin and Bones exhibit at the Smithsonian in Washington, D.C. It lets visitors use a smartphone app to visualize the skins over the animal skeletons on display.
Key to this emerging technology will be lightweight AR headsets that incorporate cameras and see-through displays. These look more like ordinary eyeglasses than those clunky VR headsets that block out the real world.
"A key capability of AR [glasses] — compared to phones — is the ability to give people contextually relevant information automatically, integrated into their perception of the physical world," says Dr. Franziska Roesner, a computer science and engineering professor at the University of Washington in Seattle.
AR could also make real the sort of interactive 3D computer interfaces seen in many sci-fi movies, says Hollywood visual effects specialist and AR designer Raffael Dickreuter, who created such effects for "Iron Man" and other films.
"We still scroll through many pages of Google and 'flat' information, whereas certain topics would be much better presented or explained in a three-dimensional, spatial environment," he says. "That's what augmented reality is bringing, and it will change many things how we get information."
Here are four ways AR will revolutionize our lives.
Behind the wheel
Many cars now come equipped with so-called "head-up" displays, which project a car's speed, GPS routes, and other information on the windshield ahead of the driver. AR glasses will take that idea further, by projecting information over your driving view — not just in front of the car but in any direction you look.
AR glasses could overlay virtual navigation beacons on the view, while keeping a watch out for pedestrians and other vehicles.
An AR system could also communicate with your car's electronics and sensors so it would know how much fuel you have left, your speed, and your braking distance. It could also flag points of interest like gas stations with 3D markers, monitor local weather and traffic reports over wireless internet, control your music player, and maybe even help you see in the dark.
On the job
By giving workers critical information when they need it — without the need to reach for a computer or smartphone — AR promises to make all kinds of work faster and safer.
At a GE Aviation manufacturing plant in Cincinnati, mechanics use Google's revamped Glass AR headsets and industrial AR software as they build jet engines. The AR system is connected by Wi-Fi to a digital torque wrench; it tells mechanics when they have the perfect fit on the many crucial B-nuts that seal an engine's fluid lines and hoses.
Medicine, too, will rapidly adopt AR technology, says Dr. Amitabh Varshney, director of the University of Maryland Institute for Advanced Computer Studies near Washington, D.C. "Some of the work we are doing in medical care and medical training will be in use within the next five years," he says, adding that doctors and nurses might begin wearing AR headsets that display vital medical data during procedures. "Think of it as an advanced F-16 head-up display for physicians," he says.
At the store
Catalog shopping will take on a new dimension with tools like Ikea Place, an iPhone app based on Apple's VR technology. It lets users scan a real-world room and then shows what it would look like with realistic images of furniture.
AR could also transform the supermarket experience: "In the near future, I think you'll see augmented reality eyewear used for shopping trips, like to the grocery store," says Varshney. "These lightweight glasses could display pricing, nutritional information, and even remind you of upcoming appointments like dropping your child off at soccer practice in 45 minutes."
One of the breakout mobile gaming hits of 2016 was the augmented reality game Pokémon Go, which saw fans by the tens of thousands chasing virtual wildlife that showed up on smartphones in certain real-world locations.
"Just like Pokémon Go, the most important applications will be the ones that will make people adopt the technology and push its advancements," says Dr. Hao Li, a University of Southern California computer scientist and CEO of mobile AR start-up Pinscreen.
And AR may soon transform a fan's experience of real-world sports, including cycling and golf. Sportwear giant Nike recently filed a U.S. patent for augmented reality golfing glasses — a wearable display linked wirelessly to a digital golf ball packed with sensors to measure its speed, trajectory, and spin rate as well as the distance it travels.
Headsets on the way
AR headsets are not for everyone yet, says Dr. Allen Yang of the Center for Augmented Cognition at the University of California, Berkeley. "Most of the applications on AR glasses are for early prototyping in field engineering, remote expert applications, and some limited medical support trials," he says.
Nonetheless, advanced AR headsets like Microsoft's HoloLens and Meta's Meta 2 are shipping to software developers, and Google's revamped Glass AR headset is going out exclusively to selected businesses through a technology partnership program.
Yang says it could take about five years for augmented reality to find wide acceptance, adding that "in 10 to 20 years, we might have an entirely new generation of young people who will be born into the world assuming AR/VR capabilities are as natural as turning on [the] TV today."