A Robot Dog Demonstrates How Mobile Manipulation Affects Automation
At a Glance:
- Boston Dynamics’ Spot delivers a new set of capabilities for manufacturers.
- The mobile platform can be mounted with a range of payloads to enhance data gathering such as thermal imaging, LiDAR or 3D scanning intended for such tasks as digital twin creation.
- The robot dog can collect limitless quantities of site data in places where humans can but shouldn’t go.
Spot was conspicuous by its absence during Marc Raibert’s keynote presentation at Automate 2022. But that didn’t stop the captive audience of robotics and automation enthusiasts from cooing over the tricks the robot dog was able to pull off in a video clip.
For the most part, attendees would have been familiar with Spot’s YouTube videos released over the years, where its dance antics and bloopers have become fodder for viral public relations campaigns. In one video celebrating the acquisition of Boston Dynamics by Hyundai Motor Group in 2021, Spot grooves with K-Pop sensation BTS. In another, the Rolling Stones team up with a pack of robot dogs that mimic Mick Jagger’s dance moves.
On the surface it seems antithetical that the Boston Dynamics founder/chairman should be on stage to sanction a dancing robot dog. Make no mistake; Raibert, a roboticist with a PhD in robotic techniques that model biological behavior, is acutely aware of the mass market appeal of the choreographed videos—where the use of legs and arms are used as a form of expression and alter perceptions of what a robot can actually do.
Humans tend to anthropomorphize robots—whether it is from Boston Dynamics or anywhere else—and public misperceptions can partly be chalked up to a lack of everyday experience with robots. “The fact is, we’re not very far along with mobile robotics doing precise operations, but we’re making progress,” said Raibert, who spent decades developing robots with advanced mobility, dexterity and perception intelligence before he designed Spot to navigate rough terrain, climb stairs or augment dangerous human work where traditional automation has failed.
What the dancing robot does demonstrate, Raibert argued, is that mobile manipulation is one of the most important robotic opportunities. He said static manipulation is proving to be successful in some kinds of manufacturing, but remains limited.
During his keynote at Collision on June 8, 2022, Raibert discussed his Today and Tomorrow robot concepts. “Spot is a general-purpose robot, and as I’ll describe, you can adapt it for a wide variety of things," said Raibert. “It’s not an appliance; it doesn’t just toast your bread, it toasts your bread, delivers your margaritas, it does a lot of things.”
READ MORE: Universal Robots’ Latest Cobot is Redesigned for Palletizing
Mobile Manipulation
Spot is a general-purpose, four-legged mobile platform that weighs 60 lb. It is compact, rugged and portable, and is designed to go where humans go. The robot dog is an omni-directional, customizable platform that can be adapted for a variety of different tasks; it can be mounted with a range of payloads to enhance data gathering such as thermal imaging, LiDAR or 3D scanning intended for such tasks as digital twin creation.
Spot has sensors built in on all four sides that help navigate through simple functions. In industrial applications, the platform may be integrated with a thermal camera, an arm for grasping objects (some people see it as a head), a Spot CAM (which comes with an optional PTZ camera), Spot CORE I/O (dedicated processing for additional on-robot computation) and Spot GXP (a power regulating and Ethernet port).
Since it moves around, tasks are no longer performed in a fixed location, and this allows businesses to think differently about the function of robotics. Singularities (a configuration in which the robot end-effector becomes blocked in a certain direction) and self-interference “are really different when you have a base that can move around,” said Raibert.
Part of Boston Dynamics go-to-market approach is a reliance on early adopters to help cultivate use cases across industries, ranging from mining, power and utilities, manufacturing, and oil and gas to NASA’s Jet Propulsion Lab. According to Raibert, there are about 1,000 Spots; 100 universities doing experiments; and about a variety of industrial facilities using Spot for such tasks as preventive maintenance inspections, remote detection, thermal, radiation and gas leak detection, and scanning facilities with a view to developing digital twins.
Perception and Intelligence
A human pilots the robot dog so it can collect data and piece together a map of the path. Spot can then be instructed to go around the path autonomously. This procedure is especially relevant where Spot is preprogrammed for “rounds and readings” in environments where routine, repeatable inspections are needed or where hundreds of pieces of equipment need to be scanned. In facilities where legacy equipment need inspection and where adding fixed sensors would not be economical, Spot can read analog gauge readings.
Raibert characterized Spot’s body manipulations for the most part as being athletic intelligence—how the robot perceives its environment and functions by moving, balancing and maneuvering around obstacles—as opposed to cognitive intelligence, which involves planning for events. Boston Dynamics has interest in leveraging both types of intelligence to bolster a fully functioning hardware system.
Operationalizing Autonomous Data Collection
Before Spot’s commercial release in 2020, Boston Dynamics turned to Rightpoint, a Chicago-based IoT consultancy, to address a gap in its user experience. “They were using a fairly bulky laptop that you had to sling over your shoulder and walk around with, and it just wasn’t a great experience to drive or control Spot,” said Ben Johnson, senior vice president of digital product, Rightpoint.
His team collaborated with Boston Dynamics in concepting, designing and developing line-of-site manipulation using either an Android tablet or a joystick to control directly where Spot was going, as well as the use of telemanipulation—or non-line-of-sight control—using the cameras to view what Spot sees in parallel with the development of Boston Dynamics’ cloud-based software (Scout).
High Fidelity Experience
From where Johnson stands, Spot is at its core an image processor and a robot that moves from Point A to Point B. “So much of the control system is the processing,” he said. Among the bugs his team needed to resolve was how to convert sensor data into a user-friendly control experience, as well as to enhance Spot’s line of sight by reimagining Spot’s camera positioning and the image the user sees. “Initially, Spot had two cameras in front that were pointing down to the ground and the two image streams intersected,” explained Johnson.
Mobile gaming provided the inspiration. “Thinking through joystick controls, we also wanted to think about a driving experience that included the ability to see the camera,” Johnson said. In addition to a straight-on view that would allow the person driving the robot to see where the robot was headed, his team would figure out how to use image processing on the tablet to transform the two image streams from the robot into an image that a human operator would expect to see. The current version, a composition of multiple images, was “a big hurdle to overcome,” said Johnson.
Thankfully, it wasn’t all work and no play. The Android tablet translates touch gestures into commands and articulations for the robot. Occasionally, those commands would get backed up and Spot would not know what to do. The results can be comedic, he said, as some of the parody videos on stress-testing can attest. “The robot just did not know where to go because the instructions it was receiving were confused or jumbled up,” Johnson quipped.
“But robotics is hard; when engineers are thinking about a connected product, so much of the challenge is in building the physical manifestation of servos and motors and positioning systems,” Johnson said. “The user experience on top of this is just a thin layer. A lot of the time you have to focus on the core mechanics, and that can be challenging.”
The robot dog has gone through many iterations since then, and Rightpoint continues to help build a safe and more robust platform. “As a society, we have gravitated to where we really do expect high fidelity experiences,” Johnson said.
Today, Tomorrow and Future Ambitions
Spot did make an appearance at IMTS 2022, where it put on a show to the delight of attendees. During the presentation, Eric Foellmer, vice president of marketing at Boston Dynamics, shared a slide depicting New York City’s Easter Parade in 1900. The procession was made up of horse-drawn carriages, except for one car. His next slide, an image taken just 13 years later, depicted the reverse: the parade was made up of a cavalcade of cars and just one horse and buggy. “The automotive industry had transformed the entire parade in a little more than a decade,” explained Foellmer, hoping to impress upon the audience that the introduction of robots into our daily lives has similarly triggered an inflection point.
“Adding cognitive intelligence to the robot is really going to be important, and I am starting to work on that problem,” said Raibert at IMTS. “That means making robots understand their surroundings. Robots should be aware of their own behavior. So, if they knock over boxes, they are going to have to figure out a plan to pick them up. I think someday, they will be able to watch another robot or a person do a task, and figure out how to do the task themselves. And they also need to use reasoning and cognitive sense.”
Raibert’s IMTS presentation in June concluded with a cliffhanger: “Stay tuned…”
Those paying attention would have guessed impending plans. In August, Hyundai and Boston Dynamics announced an initial investment of $400 million to establish the Boston Dynamics AI Institute. Located in Kendall Square research community in Cambridge, Mass., the institute will be led by Raibert with the goal to invest resources in four disciplines—namely, cognitive AI, athletic AI and organic hardware design, as well as ethics and policy.
