Pokémon GO, drone racing, and video games have people talking about augmented and virtual reality (AR and VR). While gaming is projected to take the largest part of the AR/VR market ($11.6 billion by 2025, according to Goldman Sachs), engineering and non-gaming businesses might be first to adopt this technology. As AR/VR capabilities expand into design and engineering, let’s take a look at the trends and what we may expect from the next potential computer revolution.
Applications are Key
“Everyone is concerned about the best headset and hardware associated with augmented reality, but that concern is secondary,” says Dan Arczynski, CEO of Index AR solutions. “Companies can deploy AR right now on smart phones or tablets. The difference maker is the application and how it makes the user more capable, more productive and more valuable. There is good and bad AR, and it’s not only based on the hardware.” Arczynski saw the potential in applied enterprise augmented reality when working at Newport News Shipbuilding (NNS) and Huntington Ingalls Industries. The shipyard began working with augmented reality in 2011 when it was exploring building ships without using paper drawings.
Even today as NNS continues to develop and deploy augmented reality applications, it decided not to commercialize its technology. At that point, NNS and Index signed their teaming agreement. Together, NNS and Index have developed and deployed 57 enterprise augmented reality projects over the last six years. “Augmented reality is not new technology...NNS and Index are proving its value every day,” notes Arczynski. “It will take some companies longer to understand how to use it properly, but over time AR will completely revolutionize the way a skilled worker does his or her work.”
Augmented reality is changing the way people communicate to each other, across companies, and across multiple disciplines. In the past, people would meet in one room with thousands of dollars of hardware to see an AR/VR presentation. There was no cost-effective, reliable, real time, or mobile way to conduct a presentation with this type of technology. Today, however, anyone with a smartphone could get real-time data on an AR/VR platform. This is all mainly due to the cost and size reduction in hardware. Currently, finding the best way to aggregate, distribute, and communicate data to people with this hardware is the focus.
Spatial Reasoning
Beyond bringing a group together for meetings and other functions virtually, applications abound for AR and VR. For example, it is already helping to resolve problems with spatial reasoning. VR is being used to allow people to walk around a space before it’s built. Walking in a virtual space with real walls is an interesting challenge. Many companies use a teleportation method, which allows users to select a space in the virtual room and visually teleport there. If you have ever used Street View on Google Maps and clicked around, it is the same concept.
Teleporting around a VR space helps designers manage large or complex spaces. “VR is being used as an engineering tool now,” says Shane Scranton, IrisVR’s CEO. “For example, it is a handy way for pipe fitters to model or for layout plans and building information modeling (BIM). This helps managers see everything that goes into the project, what they are paying for, and communicate to people in the field. When everyone is able to see the layout in a 3D space, it can reduce change orders and on-site alterations. This saves a lot of time and money, and is one of the drivers for AR and VR technology.”
Factory Applications
At the 2016 Rockwell Automation Fair, attendees used Microsoft’s HoloLens to work in an automated factory. It showed how the setup would look if built right on the convention floor. Attendees were able to walk around the virtual model. “The machine design market will adopt this first, because the work is already done,” notes John Pritchard, global marketing manager at Rockwell Automation. “Companies that use 3D CAD programs to develop machine models have already done about 95% of the work. All they have to do is link the virtual sensors and motors to real inputs and outputs in the controller that could take about 10 to 15 minutes. Then you streamline to the IP address of the HoloLens.” So if a company uses 3D CAD and employees have controller hardware with a connection and screen, the only thing missing for adoption is the software that connects the two. And that is available today.
Having access to data can show an augmented reality of what the future will be, and when different parts of the project are schedule to be installed, for efficient site management. This data can also be shared so a contractor can see if an on-site alteration will affect another contractor.
Once the information is sent to a device, the worker needs to be notified about the communication and be able to view it. While a headset would offer constant communication, headsets aren’t certified for eye and head protection (Z87.1 and Z89.1, respectively). DAQRI, an AR development company, focused on safety and a worker's capability when it launched its developer edition of a smart helmet. This will help engineers communicate on new levels with technicians in the field. The developer edition helmet is out right now and the company is undergoing the ANSI certification process.
Soon car gages and gadget clusters might be replaced with augmented reality projected on the windshield. This can offer data, such as speed and RPMs, but also directions and warnings about ice or blind turns.
“While we have worked on a rugged working environment for AR in BIM-type applications, this is only one small part,” says Matt Kammerait, vice president of product marketing for DAQRI. “We will see a massive increase in a heads-up applications for automobiles in the next handful of years. We already have an automotive heads-up display in over 150,000 vehicles.” AR has the potential to replace traditional gauges. In addition, such capabilities could combine with machine vision to help project images of what a driver might not be able to see during inclement conditions such as rain, fog, or around a blind turn.
One of the companies DAQRI has worked with is Autodesk. Other large software providers, such as SolidWorks and Siemens, have also been expanding software to include 3D drawings and simulations. Recently, simulations have been used to test equipment before it’s built. With the rise in AR/VR technology, finite element analysis, and simulations, Cloud services will require bigger bandwidths to transfer big data.
Data Services & The Cloud
“Forge is a set of cloud services Autodesk launched in December 2015,” explains Brian Pene, director of emerging technology at Autodesk. “The idea was to let companies build cloud apps, bring data together, and use it in ways that it hasn’t been before...People take a lot of time and money to develop their own AR/VR and data management.
“By offering a platform, companies are able to take advantage of mobile AR/VR benefits through the Cloud without worrying about the capital and time necessary to build their own,” Pene continues. “It is an interesting time, as I see all the different industries start to use cloud technologies to access data that might have previously not been available.”
Engineers can have direct communication to managers on the floor. Augmented reality can provide clear communication and access to data to reduce questions about hard-to-read 2D prints and change orders.
With more mobile data, many companies seem focused on multi-disciplinary communication. AR/VR technology is a great way to bridge the gap between the office and the shop floor. “The key is to remember that augmented reality is a disruptive technology. You can’t think of it as a traditional 2D print or drawing on a page,” notes Arczynski. “You get to communicate more authentically and visually to the worker by laying the work right over his or her field of vision.” This means knowing what the worker needs in order to do their job and providing access to data and views that might be missing on a traditional drawing, manual, or work instructions. Based on the worker and their task, the best hardware, smartphone, smart tablet or wearable, can be selected by the company.”
Don’t forget the need for audits, inspections, and documentation in multiple industries. “For example, a scheduled 36-hour ship inspection took 90 minutes when using an augmented reality application,” says Arczynski. “This AR application also achieved higher quality. My main point about AR technology first and foremost is that it is ready now. Don’t wait for a better smart device or headset or any other excuse to get started.
“Augmented reality is a revolution that will provide a competitive advantage to early adopters. As smartphones, tablets and headsets advance, augmented reality will only get more valuable.”
Source: Tech Pro Research
Some companies can see being an early adopter in a revolution as risky. Most technology moves towards increasing production while reducing cost, but companies trying to accomplish this have found a dark side. “Increasing production and reducing cost often means replacing people with automation, robotics, and/or artificial intelligence,” explains Arczynski. “Augmented reality is a direct investment in people. It helps a company’s workers be more safe, more capable and more productive.”
“We have become so limited by this idea of technology replacing workers when the better idea is to power-up their senses and extend what they’re capable of doing.” says DAQRI Founder and CEO Brian Mullins. “Other companies might argue that automation and robotics can also create jobs. By replacing low-skill labor, companies can compete and bring in new business.
“This can free up workers for more complex task that is difficult to automate. The problem then becomes, how do we get higher or medium-skilled labor?"
Virtual Training
Beyond information for daily job tasks and projects, AR and VR can help to narrow the skills gap by providing training. Every year, more engineers with decades of experience are retiring before having the opportunity to pass on their knowledge to younger engineers. One of the benefits and drivers for AR and VR technology is training. Offering a more immersive and safe training experience at a reduced cost is attractive to companies. With a headset with a camera, for example, a senior engineer can record maintenance, troubleshooting, etc., so a younger engineer can then see that video either projected over their own vision, or just have access to a video from a more experienced person walking them through a process.
Alternatively, a company can set up a digital factory. Then employees can teleport or, if space allows, walk around a machine or factory to perform tasks. Technicians will then use the same models in training to guide them on troubleshooting or maintenance in the field. If engineers know exactly what they are looking for, it would really save time in the field.
For example, AR technology could highlight hidden bolts or parts. Experienced engineers could use it to learn a new job or machine. In addition, “Visualizing data offers contextual awareness to make faster decisions,” notes Kammerait. “These faster decisions can improve safety. Data, such as how hot an object is or overhead loads, can keep workers out of the line of danger.”
Overall, education and training improves labor abilities. With AR/VR technology, it is possible to educate and train while providing access to more information that can promote skills and reduce the learning curve. Presenting complex data in an easy user-friendly way decreases training time, while providing a dynamic hands-on tool with up to date information. “AR/VR technology is saving money and attracting new customers," says Shane Scranton, IrisVR’s CEO. “This will speed up adoption, and could become a common tool by mid-2018. While many companies seem set on the potentially larger gaming market, engineering and non-gaming businesses are poised to adopt AR/VR technology before consumer.” One reason is that the Industrial IoT is connecting and granting access to more data than ever.
Industrial IoT Applications
To quickly respond to such data, the growth of AR/VR could promote remote mechatronics or control systems. Traveling to remote locations can increase cost, so having a telepresence might save time and money. Imagine a factory with a standby robot for maintenance, or the cost of shipping a robot versus a technician. The same advancements allow AR/VR data to work in real-time could communicate with telepresence devices.
With aftermarket service rising, this might be the look of a future engineer "on-location." Granted, that location might be across the world, but it could also be a VR/AR view from a robot in a small pipe where a person couldn't fit. (Click image to enlarge; courtesy Robomotive Laboratories)
For this to work, transferring massive amounts of data will be imperative. Equipment that is able to accomplish the same result with less data needing to be transferred will become valuable. Robomotive Laboratories – a robotics toy company - has found a way to accomplish this. “It takes a lot of training to pick things up with a robot. The main problem is you can’t feel what you are picking up,” said Brett Pipitone, CEO of Robomotive Laboratories. “Companies have used force feedback sensors, but I felt the math required for this approach made the processing power too great to support any type of reasonable cost. I used Position feedback force control. It sends the position of the gripper to the controller, and the force the user is applying to the controller to the gripper. This is an inexpensive way to feel what you are picking up while taking up less processing power.”
While this application is a toy, Pipitone said the technology can be used for industrial uses. This remote technology could be used in hazardous areas, or even help the daily fight in the spread of germs. In the healthcare arena, touch screens, keyboards, and other device offer important data, but may spread germs because multiple people use them. AR technology can use gesture recognition to interact with data projected in the medical professional’s view without having him or her touch an HMI.
AR/VR technology goes beyond communicating between people, yet its potential is still in its infancy as we discover how to create, implement, and apply this technology. Already, researchers used VR technology to help paralyzed patients train their brain to control an exoskeleton. The hope was to use brainwaves to control an exoskeleton so patients could walk. Not only was this possible, but some patients regained some feeling and movement to their legs. AR/VR technology is helping to communicate across disciplines and neuropathways. In this video, the patient’s movements look like a small twitch, but it’s a giant leap for an AR/VR revolution.
