Pokémon GO, Drone racing, and video games have people talking about augmented and virtual reality (AR and VR, respectively). While gaming is projected to take the largest part of the AR/VR market—11.6 billion by 2025, according to Goldman Sachs—enterprise might be first to adopt this technology. As AR/VR markets expand into engineering, let’s take a look at what to expect from the next potential computer revolution.
Past AR
“Everyone is concerned about the best headset, and hardware associated with the technology, but that is the wrong focus,” said Dan Arczynski, CEO of Index AR solutions. “People can use AR right now with their smartphone. The difference is the software and how it is communicating with the users. There is good and bad AR, and it’s not based on the hardware.” Arczynski saw the potential in the software when working for Newport News Shipbuilding (NNS). The company wanted to build a new deck plate for a boat without using paper prints.
After NNS developed an AR technology, the company expressed no interest in commercializing it, so Arczynski entered into an agreement and formed Index AR Solutions. Since then he has used AR technology in some 57 projects over the last six years. “This is not new technology, said Arczynski. “It will just take some companies longer to see how to use it properly. AR will completely revolutionize the way manufacturing skilled labor is used.” It is changing the way people communicate to each other, both across companies and disciplines. For a company to do this right, it will take a paradigm shift to integrate such a disruptive technology.
Processing power and hardware limits are no longer a problem. In the past, people would meet in one room with thousands of dollars of hardware to see an AR/VR presentation. There was no cost-effective, reliable, real-time, or mobile way to conduct a presentation with this type of technology. Today however, anyone with a smartphone can get real-time data on an AR/VR platform. This is all mainly due to the cost and size reduction in hardware. Currently, finding the best way to aggregate, distribute, and communicate data to people with the hardware is the focus.
Current AR/VR
Spatial reasoning. Spatial reasoning can be difficult. This is one way VR is being used to allow people to walk around a space before it’s built. Walking in a virtual space with real walls is an interesting challenge. Many companies use a teleportation method, whereby users will select a space in the virtual room, and visually teleport there. If you have ever used Street View on Google Maps and clicked around, it is the same concept.
Augmented reality can show data at a glance. Inventories, part numbers, costs, and more can be accessed in a user-friendly way. More information can help make better decisions. Having the information at a glance can make better decisions faster.
Teleporting around a VR space helps designers manage large or complex spaces. “VR is being used as an engineering tool now,” said Shane Scranton, IrisVR’s CEO. “For example, it is a handy way for pipe fitters to model or layout plans and building information modeling (BIM). This helps managers see everything that goes into the project, what they are paying for, and communicate to people in the field.
“When everyone is able to see the layout in a 3D space, it can reduce change orders and on-site alterations,” Scranton added. “This saves a lot of time and money, and is one of the drivers for AR and VR technology.”
BIM. VR technology was presented at the 2016 Rockwell Automation Fair. Attendees used Microsoft’s HoloLens to work in an automated factory. It showed how the setup would look if built right on the convention floor, and attendees were able to walk around the virtual model.
“The machine design market will adopt this first, because the work is already done,” said John Pritchard, Rockwell Automation’s global marketing manager. “Companies that use CAD programs to develop prints have already done about 95% of the work. All they have to do is link the virtual sensors and motors to real inputs in the controller that could take about 10 to 15 minutes. Then you send the software to the IP address of the HoloLens, smartphone, etc.”
If a company uses a CAD program, and employees have some type of hardware with a connection and screen, the only thing missing for adoption is the software, and that is available today.
The display on the right shows what the person on the left with the HoloLens sees as he looks around a factory floor. The HoloLens can use hand gestures to open menus, or launch models for a more inclusion free environment.
Once the information is sent to a device, it will be important that the worker sees the communication. A worker might not be able to hear a notification, or have a free hand to grab for a phone or tablet. While a headset would offer constant communication, headsets aren’t certified for eye and head protection (Z87.1 and Z89.1, respectively).
DAQRI, an AR development company, focused on safety and a worker’s capability when it launched its developer edition of a smart helmet. This will help engineers communicate on new levels with technicians in the field. DAQRI’s developer edition helmet is out right now, allowing these workers to go through the ANSI certification process.
“While we have worked on a rugged working environment for AR in BIM-type applications, this is only one small part,” said Matt Kammerait, vice president of marketing for DAQRI. “We will see a massive increase in heads-up applications for automobiles in the next handful of years. We already have an automotive heads-up display in over 150,000 vehicles.” AR has the potential to replace traditional gages. In addition, this could work with machine vision to help project images of what a driver might not be able to see during inclement conditions such as rain or fog, or around a blind turn.
Soon car gages and gadget clusters might be replaced with augmented reality projected on the windshield. This can offer data, such as speed and RPMs, but also directions and warnings about ice or blind turns.
CAD. One of the companies DAQRI has worked with is AutoDesk. Other large software providers, such as SolidWorks and Siemens, have also been expanding software to include 3D drawings and simulations. Simulations have increasingly been used to test equipment before it’s built. Now with the rise in AR/VR technology, finite element analysis, and simulations, Cloud services will require bigger bandwidths to transfer big data.
“Forge is a set of cloud services AutoDesk launched in December 2015,” said Brian Pene, director of emerging technology and CTO at Autodesk. “The idea was to let companies build cloud apps, bring data together, and use it in ways that it hasn’t been before.
“People take a lot of time and money to develop their own AR/VR and data management,” Pene added. “By offering a platform, companies are able to take advantage of mobile AR/VR benefits through the cloud without worrying about the capital and time necessary to build their own. It is an interesting time, as I see all the different industries start to use cloud technologies to access data that might have previously not been available.”
Engineers can have direct communication with managers on the floor. Augmented reality can provide clear communication and access to data to reduce questions about hard-to-read 2D prints and change orders.
With more mobile data, many companies seem focused on multi-disciplinary communication. AR/VR technology is a great way to bridge the gap between the office and the shop floor. “With this technology, the key is to remember: this is a disruptive technology,” said Arczynski. “You can’t think of it as a traditional 2D print, or 3D image on a page.
“You get to communicate more authentically and from the workers prospective by laying the work right over the laborers field of vision,” he added. This means knowing what the worker needs, and providing the worker access to data and views that might be missing on a traditional print. Minimum interference with the worker’s vision while providing maximum communication is the goal. This doesn’t mean finding the best headset to present the data. With the right software, a worker could use their phone or a tablet.
Training. Another gap companies are concerned with is the experience or skills gap. Every year, more engineers with decades of experience are retiring before having the opportunity to pass on their knowledge to younger engineers. One of the benefits and drivers for AR and VR technology is training. Offering a more immersive and safe training experience at a reduced cost is attractive to companies. Offering something like a headset with a camera lets a senior engineer record maintenance, troubleshoot, etc. A younger engineer can then see that video, either projected over their own vision or via other access.
Lacking an engineer with experience to video a process, a company can set up a digital factory. Then employees can teleport or, if space allows, walk around a machine or factory to perform tasks. Technicians will then use the same models in training to guide them through troubleshooting or maintenance. When engineers know exactly what they are looking for, this could save time in the field.
AR technology could highlight hidden bolts, or parts. Experienced engineers could use it if it’s a new job or new machine. In addition, “Visualizing data offers contextual awareness to make faster decisions,” said Kammerait. “This data might not only improve training, but also safety. Connecting more devices through IoT, smart sensors, etc., workers might be able to avoid injury. Seeing how heavy or hot an object is before touching it may reduce injuries, or more complex data could keep workers out of the line of danger.”
Audits. Audits, inspections, and documentation play a key role in multiple industries. I haven’t met anyone that enjoys this part of the job, but sometimes it is necessary. Connecting with technicians and engineers in this shared space can streamline documentation, and therefore reduce inspection and audits.
“For example, a scheduled 36-hour ship inspection took 90 minutes using augmented reality,” said Arczynski. “It was also performed safer while achieving a higher quality. The main point about AR/VR technology is that it is ready now. If you think it’s not, it’s because you are thinking about which headset is better. Augmented reality is a revolution, with many benefits for the early users in industry.”
Future AR/VR. Most technology focuses on increasing production while reducing cost. “The problem is, most technology has a dark side,” said Arczynski. “This sometimes means replacing people with automation and robotics. AR/VR technology is an investment in people. I think this is why it is important to invest in AR/VR for the future. It can help people be more productive while reducing costs—not jobs.”
“We have become so limited by this idea of technology replacing workers, when the better idea is to power-up their senses and extend what they’re capable of doing,” added Brian Mullins, founder and CEO of DAQRI.
Other companies might argue that automation and robotics can also create jobs. By replacing low-skill labor, companies can compete and bring in new business. This can free up workers for more complex tasks that are difficult to automate. The problem then becomes, how do we get higher- or medium-skilled labor?
Education and training can help improve labor. With AR/VR technology, it is possible to educate and train while providing access to more information that can promote skills and reduce the learning curve. Presenting complex data in an easy, user-friendly way decreases training time, while providing a dynamic hands-on tool with up-to-date information.
“AR/VR technology is saving money and attracting new customers,” said Shane Scranton, IrisVR’s CEO. “This will speed up adoption, and could become a common tool by mid-2018. While many companies seem set on the potentially larger gaming market, enterprise is poised to explode or adopt AR/VR technology before consumer.”
IIoT. In an enterprise market, more things are becoming increasingly connected. Connected devices are increasing the importance of access and presentation of data. Graphics and images communicate across disciplinary lines and offer quick access and understanding to data. For Millennials who grew up with more technology than previous generations, this type of technology may become not just more welcome, but necessary to continue fast, effective communication.
The growth of AR/VR could promote remote mechatronics or control systems. Traveling to remote locations can increase cost, so having a tele-presence might save time and money. Imagine a factory with a standby robot for maintenance, or the cost of shipping a robot versus shipping a technician. The same advancements allowing AR/VR data to work in real-time could communicate with a tele-presence device.
Aftermarket and service industries are increasing. Having as an on-site IIoT robot that engineers can access and control of with AR/VR data could be beneficial. For this to work, transferring massive amounts of data will be imperative. Equipment that is able to accomplish the same result with less data needing to be transferred will become valuable.
While Robomotive Laboratories—a robotics company—has a tele-presence device, it’s more of a toy. However, they have found a way to use less bandwidth. “It takes a lot of training to pick things up with a robot,” said Brett Pipitone, the firm’s CEO. “The main problem is you can’t feel what you are picking up. Companies have used force feedback sensors, but I felt the math required for this approach made the processing power too great to support any type of reasonable cost.
“I used Position feedback force control,” Pipitone added. “It sends the position of the gripper to the controller and the force the user is applying to the controller to the gripper. This is an inexpensive way to feel what you are picking up, while taking up less processing power.”
With more advanced tele-presence devices, many industries could change. However, it is all just another way to communicate with one another. In the healthcare industry, proper communication and preventing the spread of germs is important. Touch computer screens, keyboards, and other devices offer important data, but may circumvent the spread of germs. With AR technology, another future application could present the data as an AR in the medical professional’s vision. With gesture recognition, a medical professional could access the information needed without touching anything.
AR/VR technology has been finding many uses, and in some cases is showing unbelievable results. Researchers used VR technology to help paralyzed patients train their brain to control an exoskeleton. The hope was to use brainwaves to control the exoskeleton so patients could walk. Researchers noticed after testing that not only was this possible, but some patients regained some feeling and movement in their legs. To us it might look like a small twitch, but it’s a giant leap for VR research.
