As technology advances, robots have also made remarkable progress in terms of their capabilities. However, the challenge of navigating highly congested spaces, such as busy public areas and bustling urban roads, still poses a significant obstacle for robots. To truly integrate robots into our daily lives, especially in the smart cities of the future, they must be able to move around in these environments with both safety and reliability in mind. This means developing innovative solutions to ensure that robots can seamlessly and smoothly navigate these crowded areas without the risk of colliding with people or objects nearby.
Some really smart people in Spain have figured out a new way to help robots move around in crowded places like inside buildings or busy streets. They came up with a special kind of reward that helps the robot learn how to explore new areas and move around in different ways. By doing this, the robot can learn how to navigate the space better and avoid bumping into people or things. It’s like giving the robot a special treat for trying new things, which helps it become a better navigator.
“Autonomous robot navigation is an open unsolved problem, especially in unstructured and dynamic environments, where a robot has to avoid collisions with dynamic obstacles and reach the goal,” Diego Martinez Baselga, one of the researchers who carried out the study, told Tech Xplore. “Deep reinforcement learning algorithms have proven to have a high performance in terms of success rate and time to reach the goal, but there is still a lot to improve.”
Martinez Baselga and his colleagues developed a new way to help robots learn how to navigate in crowded places. They did this by giving the robots rewards when they explored new areas, which made them better at predicting what would happen when they took action in those areas. This could help robots move around safely and effectively in the future.
“Most of the works of deep reinforcement learning for crowd navigation of the state-of-the-art focus on improving the networks and the processing of what the robot senses,” Martinez Baselga said. “My approach studies how to explore the environment during training to improve the learning process. In training, instead of trying random actions or the optimal ones, the robot tries to do what it thinks it may learn more from.”
Martinez Baselga and his team tested how well intrinsic rewards could help robots navigate crowded spaces by trying out two different ways. The first method is called an “intrinsic curiosity module” (ICM), which helps the robot be curious and explore its environment. The second method is based on a set of algorithms called random encoders for efficient exploration (RE3), which help the robot explore its surroundings more efficiently. The researchers wanted to see which method worked better in helping the robot move around in crowded spaces without causing any harm. By testing these two approaches, they hope to find the best way to help robots navigate in the future.
The scientists tested their new idea on a computer program that acts like a big video game. They used two different ways to teach a virtual robot how to move around in crowded areas, like a busy street. The first way was to make the robot curious and explore new things in the game. The second way was to help the robot explore the game more efficiently. The scientists found that both of these new ways worked better than the old ways to teach the robot how to move around without bumping into anything or anyone. It’s like they found a new cheat code that makes the robot really good at navigating through a crowd!
Martinez Baselga and his team’s research could inspire other robot builders to use the new idea of intrinsic rewards when teaching their robots. This could help robots become better at handling difficult situations and moving safely in really busy places. The two new methods they developed using intrinsic rewards could also be tested on real robots in the future to see if they work just as well as they did in the computer program. This could help make robots safer and more helpful to people in the real world.
“The results show that applying these smart exploration strategies, the robot learns faster and the final policy learned is better; and that they could be applied in top of existing algorithms to improve them,” Martinez Baselga added. “In my next studies, I plan to improve deep reinforcement learning in robot navigation to make it safer and more reliable, which is very important in order to use it in the real world.”
You might also be interested in reading, A machine learning algorithm is being used to predict city traffic
