human-robot interaction

Most often, humans and robots do not have to work directly together, instead working on different parts in a production pipeline or with the robot performing tasks instead of a human. In such cases any human-robot interaction (HRI) will be superficial. Yet what if humans and robots have to work alongside each other? This is a question which a group of students at MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) have recently studied some answers to.

In their paper on human-robot collaborative tasks (PDF), they cover the three possible models one can use for this kind of interaction: there can be no communication (‘silent’), the communication can be pre-programmed (state machine), or in this case a Markov model-based system. This framework which they demonstrate is called CommPlan and it uses observation data from human subjects to construct a Markov model that can integrate sensor data in order to decide on its next action.

In the experiment they performed (the preparation of a meal; see the embedded video after the break), human subjects had to work alongside a robot. Between the three different approaches, the CommPlan one was the fastest, using voice interaction only when it deemed it to be necessary. The experiment’s subjects expressed hereby a preference for bidirectional communication, much as would occur between human workers.

Continue reading “Teaching Robots Workplace Etiquette” →

New framework enables human-robot interaction for broader access to deep sea scientific exploration

Scientific exploration of the deep ocean has largely remained inaccessible to most people because of barriers to access due to infrastructure, training, and physical ability requirements for at-sea oceanographic research.

Now, a new and innovative framework for oceanographic research provides a way for shore-based scientists, citizen scientists, and the general public to seamlessly observe and control robotic sampling processes.

The Shared Autonomy for Remote Collaboration (SHARC) framework "enables remote participants to conduct shipboard operations and control robotic manipulators"—such as on remotely operated vehicles (ROVs)—"using only a basic internet connection and consumer-grade hardware, regardless of their prior piloting experience."

This is according to a paper in Science Robotics, titled "Enhancing scientific exploration of the deep sea through shared autonomy in remote manipulation." The framework has been developed by a research team from the Woods Hole Oceanographic Institution (WHOI), the Massachusetts Institute of Technology (MIT), and the Toyota Technological Institute at Chicago (TTIC).

The SHARC framework enables real-time collaboration between multiple remote operators, who can issue goal-directed commands through simple speech and hand gestures while wearing virtual reality goggles in an intuitive three-dimensional workspace representation.

Through SHARC, "we can open up the operational aspects of deep sea exploration to citizen scientists, whether they be kids in a classroom or people who can't be present on a ship because of logistical or physical requirements," said co-author Richard Camilli, a principal investigator for the project and scientist in WHOI's Applied Ocean Physics and Engineering Department. "Citizen scientists can interact with the ROV's robotic manipulator arm in a virtual world, somewhat analogous to the science fiction 'holodeck' holographic system used on Federation starships on Star Trek."

SHARC enables human-robot interaction—sometimes referred to as shared autonomy—that delegates responsibilities between the robot and the human operator based on their complementary strengths. The robot, for instance, can handle kinematics, motion planning, obstacle avoidance, and other low-level tasks, while human operators take responsibility for high-level scene understanding, goal selection, and task-level planning. In addition, SHARC allows for parallel rather than sequential operation.

"We just give the robot its goal, and it finds a solution," said Camilli. "People and the robot can collaborate together, where we're not waiting for one thing to happen in order to do the next thing. While the robotic arm is executing a task, we can be focusing on the next goal."

In September 2021, during the height of the COVID pandemic, scientists successfully tested SHARC. During an oceanographic expedition in the San Pedro Basin of the Eastern Pacific Ocean, SHARC team members operated WHOI's Nereid Under Ice (NUI) hybrid ROV from thousands of kilometers away using SHARC's virtual reality and desktop interfaces.

The team members—physically located in Chicago, Boston, and Woods Hole—collaboratively collected a physical push core sample and recorded in-situ X-ray fluorescence measurements of seafloor microbial mats and sediments at water depths exceeding 1000 meters. Science Robotics (2023). DOI: 10.1126/scirobotics.adi5227

"This paper really highlights shared autonomy's potential to help democratize access to the deep sea," said lead author Amy Phung, who is a student in the MIT-WHOI Joint Program in Oceanography/Applied Ocean Science and Engineering graduate degree program. Phung was one of the scientists operating the NUI vehicle during the 2021 test of SHARC.

"With SHARC, our shore-side team was able to collect seafloor samples from over 4000 kilometers away without specialized hardware or extensive prior training. In the future, I believe that further advancements in robotics and autonomy research can someday enable shore-side scientists, students, and enthusiasts to actively participate in and contribute to deep-ocean exploration operations as they occur, which in turn can help to foster ocean literacy among the general public," Phung added.