An efficient—and rather dramatic—method for placing a swarm of rescue or research drones has been devised by Skoltech researchers. To direct each drone toward its desired location with a succession of shots, the operator dons a virtual reality helmet and a tactile interface to simulate shooting a bow.
Deep reinforcement learning improves the method and helps to keep the robots from colliding with one another. The research is the first Russian initiative to be chosen for this prestigious conference since its start more than 20 years ago, and it will be presented at the 21st IEEE International Symposium on Mixed and Augmented Reality.
“When he promoted touch displays and gestures, Steve Jobs fundamentally altered our conception of a natural engagement with the digital world. Our research’s aim is even more ambitious because it aims to tackle the problem of how to make it possible for a user with no prior drone piloting experience to quickly deploy an entire swarm “Director of the Intelligent Space Robotics Laboratory at Skoltech Associate Professor Dzmitry Tsetserukou commented on the study.
Lead author of the research and recent Skoltech MSc graduate Ekaterina Dorzhieva said, “Right now, there aren’t that many interfaces for deploying a swarm of drones. “When operating one drone, a joystick is easy, but when managing a whole swarm, you either need many operators or complicated software with code that explicitly accounts for a lot of things in an unintuitive manner. We provide a counterargument to this.”
The alternative option offered by the Institute’s Engineering Center is a tactile interface developed by Miguel Altamirano Cabrera, a PhD student at Skoltech, and motivated by LinkTouch, an earlier concept made by Tsetserukou in Japan.
The user is outfitted with a tactile interface, a virtual reality helmet, and gloves with markings. The “shooter” is surrounded by a motion capture system that has numerous IR cameras to monitor the placement of the gloves and drones. The tactile interface enables the user to control the distance to the spot they are going to deploy a drone by feeling the tension of the virtual bowstring depending on how far apart the gloves are from one another.
The VR helmet continues to display the drone’s ballistic trajectory in real-time in the meanwhile. The operator opens their fist once they are pleased with the trajectory. The camera detects this, locks the trajectory, and commands the drone to go along it. Once the drone has arrived at its destination, some conventional form of control must take over and direct it on its mission. This mission could involve a rescue operation, natural resource management, crop monitoring using the vegetation index, pollution detection, infrastructure inspection, or maintenance.
“The advantage of employing a ballistic trajectory that resembles an arrow is that a human operator can rapidly discover a method to deploy drones while avoiding obstacles since it seems natural to them. It’s a typical human activity, “explained Dorzhieva. “You could use software to programme the locations for drones to go there, but they would then have to determine the route that avoids the obstacles on their own, which requires each machine to be outfitted with a camera.”
This has two more turns. The operator does not need to be in close proximity to the actual place where the swarm is being deployed when employing virtual reality technology, for starters. The VR helmet might be used to replicate operator presence on site and place the robots in any challenging environment that the drones are designed to resist, such as a raging fire, radioactive pollution, cold temperatures in a distant area, etc.
Additionally, there is undoubtedly promise for the entertainment and video gaming industries if virtual reality and bow shooting are combined. Second, the new approach is strengthened by reinforcement learning, allowing the drones to anticipate potential collisions and modify their current trajectories as necessary.
