George Lucas’s 1977 space opera, "Star Wars: Episode IV – A New Hope," unleashed a special effects revolution, conspicuously manifested in a hologram of Princess Leia making a linchpin plea: “Help me, Obi-Wan Kenobi. You’re my only hope.”
Visually propelling the story along, holography seemed to blur the lines between sci-fi and reality. But while the hologram’s relevance as a form of a real-time visual communication is a hallmark of the Star Wars saga, its function and appeal resonate well beyond the realm of science fiction.
Holography experts would have you know that the iconic projection of Princess Leia is, in fact, a misrepresentation of a hologram. A 3D image that floats in air—one that you can walk around and see from all angles—is called a volumetric image. “Whereas a holographic display scatters light only at a 2D surface, a volumetric display has little scattering surfaces scattered throughout the 3D space occupied by the 3D image,” said Daniel Smalley, a Brigham Young University electrical and computer engineering professor and holography expert, in a paper published in Nature.
Not to be confused with virtual reality (VR), which simulates a real-life environment and takes users out of their setting, augmented reality (AR) layers digital information (graphics, imagery and objects) atop the user’s physical surroundings to render the illusion that the digital information coexists in the same space.
Today, various forms of AR—whether it’s rendered through a hologram, volumetric image or high-tech adaptation of the Pepper’s Ghost stage trick—are unlocking the transformative potential of immersive, sensory experiences, including the beaming of a keynote speaker, a CEO or professor to a live audience halfway around the world in real time.
Holographic Telepresence
Toronto-based ARHT Media harnesses its augmented reality holographic technology to solve the problem of getting subject matter experts to meetings they couldn’t get to otherwise. The platform has captured the speaker market by delivering realistic representations of high-profile celebrities and presenters—the likes of Deepak Chopra, Tony Robbins, Warren Buffett, Gary Vaynerchuk and Stephen Hawking—to live events.
ARHT’s CEO, Larry O’Reilly, explained that ARHT’s holographic telepresence technology uses a 3D photographic technique to project a lifelike image (“HumaGram”) that’s visible to an audience at another location without the use of 3D glasses. “Think of a TV weatherperson in front of a green screen or black screen, with some lighting, a microphone, wearing an earpiece,” explained O’Reilly, formerly the president of worldwide sales at IMAX Corp. “We capture that person’s image using a Sony 4K commercial camera and audio with the microphone, and send that information to our server. That’s where our technology stack is centered.
“The data is then encrypted and compressed, and we can send it anywhere in the world to media servers,” he continued. “We can beam simultaneously at multiple locations. Then, depending on the venue, audio feeds to the audio system. The video feed goes to a projector, which projects the image onto a mesh that is painted with a proprietary highly reflective paint. A second plane behind that screen shines light on either side of the subject.”
From an audience perspective, the life-size hologram appears in high resolution. Lights are purposefully positioned behind the screen, so your eyes fuse the two images to create the illusion of 3D. To complete the illusion, ARHT’s crew adjusts the audio to transmit sound with less than 0.3 seconds of latency.
ARHT works in a variety of settings—boardrooms, lecture halls, concert arenas—all of which require different types of staging and technical expertise, said O’Reilly. “Being able to create a lifelike illusion and overcoming latency are crucial. If it appears as a video, that human connection is lost.”
Social Interaction
What makes the ARHT platform dynamic O’Reilly noted, is the ability to deliver a sensation that you’re speaking to the person in the same room, whether it’s in front of an audience of 10,000 in an arena, or one-on-one in a business meeting. Stanford researchers studying the effects of AR on social behavior found that people adjusted their interactions in the physical world after an AR interaction.
For instance, participants in one of three studies conducted avoided sitting on a chair that was previously occupied by a virtual person in a similar way that they would be if a real person were next to them. The studies aren’t without limitations, as the samples for all three studies were mostly composed of undergraduate students with little demographic variance, but the research nonetheless suggests that task performance, nonverbal behavior and social connectedness are “significantly affected” by the presence or absence of virtual content.
“Your brain tells you that they’re in the room with you,” said O’Reilly. “That’s how the emotional connection actually happens; that’s the power of the technology.”
Inflection Point
Had the Mobile World Conference in Barcelona not been cancelled due to global concerns over the coronavirus outbreak in February, O’Reilly’s own hologram could have served as a substitute for his actual appearance at the trade show. The logistics of being digitally transported (“holoported”) is more cost-effective than actually being there, he said.
In fact, O’Reilly regularly beams into weekly meetings at his company’s offices in London, New York and Los Angeles from the Toronto office. “Instead of having to get on a plane, I can beam live into either Singapore or Hong Kong, all on the same day and have the same impact as being face-to-face with people,” he said.
As O’Reilly sees it, the shift in the number of organizations with major initiatives to reduce their carbon footprint, as well as a cultural shift whereby executives want to spend more time at home, is an opportunity to help them achieve their goals. Moreover, being cognizant of the carbon footprint is an inflection point that’s causing global enterprises to lean in.
O’Reilly cited the stunning shift by BlackRock CEO, Larry Fink, who warned in his annual letter to CEOs (published Jan. 14) that an intensifying climate crisis would bring about a “fundamental reshaping of finance” going forward. The New York-based asset management firm explained that it would avoid investments in companies with high sustainability-related risk.
Professional services companies on ARHT’s client roster have noted that 84% of their carbon footprint are from executives flying to meetings. “Well, if they can take 25%, 30%, 40% of those meetings holographically, then the reduction in carbon footprint is massive,” said O’Reilly, adding that, in the fourth quarter of 2019, about 30% of ARHT’s inbound inquiries listed their No. 1 or No. 2 reasons for calling was to reduce their carbon footprint. “Since the World Economic Forum in Davos, it’s probably over half.”
Competitive Strength
While ARHT is focused on the business side of entertainment (which is to say, conferences), prospective clients regularly inquire whether it would consider alternative applications for its platform. At present, the platform is active in four verticals: entertainment, healthcare, financial services and the tech industry.
Yet, application opportunities remain boundless. There is an unstoppable momentum behind 5G in the tech space as the anticipated use for medical imaging, video conferencing and gaming for 3D holographic communication requires about four times as much data as a streamed 4K video.
But only 5G has the requisite speed and low latency to support these kinds of applications, according to Swedish telecom Ericsson. The firm counts its Lewisville factory and an advanced 5G technology manufacturing facility in North Texas (slated to open later this year) among a string of investments in the 5G market.
In a pioneering collaboration between Ericsson and Vodafone, a reporter from a national news program appeared at one of Portugal’s largest musical festivals (Vodafone Paredes de Coura) in HD hologram form over a 5G network. The reporter was able to interact with the CTO of Vodafone, João Nascimento, in 3D with minimal transmission latency. Ericsson reports consumers are very interested in these kinds of use cases and predicts that 5G augmented events and 3D hologram calling are among services that will be mainstream within five years.
Similarly, when used in healthcare, biotechnology and big pharma, holograms used in image-guided therapies are disrupting the way medical professionals plan and carry out treatments and surgeries. In a breakthrough example, scientists at SLAC National Accelerator Laboratory and Stanford University successfully used a holographic method called in-flight holography to demonstrate the first X-ray holograms of nano-sized viruses that were not attached to any surface.
Nanoviruses had been studied at the lab without a holographic reference, but the interpretation of the X-ray images required many steps, relied on human input and was a computationally challenging task. To overcome some of these inefficiencies, the scientists developed a two-step procedure whereby scattered X-ray light from the virus was superimposed with scattered X-ray light from a reference nano-sized sphere. They explained that the curvature in the superimposed images from the two objects provided depth information and details about the shape of the 450-nanometer-wide virus, the minivirus.
This technique greatly simplified the interpretation of the data. Before the study, the structure of nanosamples was reconstructed long after the actual experiment, but with “in-flight” holography the procedure was simplified and, in principle, can be performed while taking data, noted the researchers. In the long run, they predict that in-flight holography will offer new ways to study air pollution, combustion and catalytic processes.
First-Mover Advantage
With crews readily accessible in Los Angeles, New York, Toronto, London, Hong Kong and Fuzhou, China, as well as affiliates in Tel Aviv and Seoul, ARHT continues to work on lowering the cost of delivery. Fortunately, 90% of the equipment ARHT uses are readily available, said O’Reilly.
The typical challenges associated with scaling up a business—size, geographical and administrative scalability—tend to be less of a concern. “The challenge in scaling is that we’re educating the market in new ways of communicating, and once major global brands use the technology, they start to look to adopt it. Change in behavior takes time,” said O’Reilly.
For O’Reilly, the prospect of receiving requests for having ARHT's holographic technology applied across large corporations for day-to-day conferencing isn’t inconceivable. “If companies say, ‘we’ve used it five times, 10 times or 15 times, let’s just put it everywhere,’ then it’s going to require some focus and some energy to actually deliver,” O’Reilly said.
“What we’re deploying is expertise and our software to enable the communication. And that’s very scalable.”
Citation
Miller MR, Jun H, Herrera F, Yu Villa J, Welch G, Bailenson JN (2019) Social interaction in augmented reality. PLoS ONE 14(5): e0216290. https://doi.org/10.1371/journal.pone.0216290
