A small drone successfully learned to see in zero-gravity by to see distances using only one eye. Researchers presented the preliminary results of this experiment at the 67th International Astronautical Congress (IAC) in Guadalajara, Mexico.
Although humans effortlessly estimate distances with one eye, it is not clear how we learn this capability, or how robots should learn the same. It is a mathematical impossibility to extract distances to objects from one single image, as long as one has not experienced the objects. But once we recognise something to be a car, we know its physical characteristics and we may use that information to estimate its distance from us. A similar logic is what we wanted the drones to learn during the experiments.
Stereo vision information recording
During the experiment, a drone started navigating in the ISS while recording stereo vision information on its surroundings from its two eyes (cameras). Then it started to learn about the distances to walls and obstacles encountered so that when the stereo vision camera would be switched off, it could start an autonomous exploratory behaviour using only one eye (a single camera).
The drone’s learning approach based on self-supervised learning, which has a high degree of reliability and helps drone autonomy. A similar learning approach was successfully applied to self-driving cars, a task where reliability is also of paramount importance.
Drone in zero-gravity
It was very exciting to see, for the first time, a drone in space learning using cutting edge AI methods, added Dario Izzo who coordinated the scientific contribution from ESA’s Advanced Concepts Team. At ESA and in particular here at the ACT, we worked towards this goal for the past 5 years. In space applications, machine learning is not considered as a reliable approach to autonomy a bad learning result may result in a catastrophic failure of the entire mission. Our approach, based on the self-supervised learning paradigm, has a high degree of reliability and helps the drone autonomy a similar learning algorithm was successfully applied to self-driving cars, a task where reliability is also of paramount importance.
The experiment was designed in collaboration between the Advanced Concepts Team (ACT) of the European Space Agency (ESA), the Massachusetts Institute of Technology (MIT) and the Micro Air Vehicles lab (MAV-lab) of Delft University of Technology (TU Delft), and was the final step of a five-years research.