Recently, researchers at Facebook and the University of California, Berkeley have developed a way to make drones fly “with a load.” According to the simulation results, the drone can pick up, transport, and unload the payload while maintaining a smooth flight state.
There has long been a desire to use drones to deliver goods in warehouses or other Industrial settings. But previous research has shown that carrying a payload can impair the drone’s flight performance and may even cause the drone to malfunction. In this study, the researchers creatively used meta-learning methods to solve this problem. It is understood that this is the first time that a meta-learning method has been used to solve the problem of unmanned aerial vehicles.
The research, published on the academic website arXiv, is titled “Model-Based Meta-Reinforcement Learning for Flight with Suspended Payloads”.
Paper link: https://arxiv.org/pdf/2004.11345.pdf
1. Meta-learning: Let the model draw inferences from experience
Loading heavy objects can have unpredictable effects on the drone’s dynamics model. Previous studies have attempted to solve the problem with adaptive control and learning methods, but these methods have some limitations.
First, modeling is very difficult due to the complexity of the UAV operating environment. In an industrial environment, the quality of the cargo that needs to be delivered by a UAV is not a priori, and modeling an isolated physical state does not help the UAV to adapt to all situations, so a manually designed dynamic model is not sufficient for online control.
Additionally, machine learning models often require a lengthy data calibration process. But during the flight of the drone, it must adapt quickly after picking up the payload, otherwise it may deviate from the flight path or even have a serious failure.
To address these issues, researchers at Facebook and UC Berkeley have proposed a model-based approach to meta-reinforcement learning. “Meta Learning” is a machine learning method that can improve the learning efficiency of the model and let the model “learn how to learn”.
The researchers trained a dynamic prediction model based on a deep neural network to help the drone adapt to different payloads, and used a quadcopter drone to carry a payload of unknown weight to verify the model effect.
2. Predictive model: help drones to continuously optimize flight actions
The neural network dynamics prediction model takes the current state and actions of the drone as input. When the UAV is loaded with a payload, the prediction model uses the variational inference method to quickly infer the posterior probability according to the state parameters such as the mass of the current payload and the length of the tether, so as to help the UAV adapt to the new flight status.
The model is trained on time series data of length T to optimize the weight parameters of the neural network. The researchers assumed that the payload parameters were unknown, represented them by a latent variable K with distribution parameters, and adjusted the value of K to simulate carrying different payloads.
During the model training phase, the researchers manually flew drones carrying different payloads along random trajectories and collected this part of the training data. The researchers then ran a meta-learning approach that allowed the model to learn the shared dynamics model parameters and adaptation parameters for different payloads.
The researchers then examine the training outcomes of the model. The model derives the optimal latent variables online using all the data for the current task. The controller based on the dynamic model plans the action of the UAV accordingly, so that the UAV flies according to the predetermined route. During the entire flight process, the model will continue to store data, continuously derive the optimal latent variables, and optimize the drone’s actions until it reaches the destination.
3. UAVs can carry loads to complete avoidance, loading and unloading tasks
The researchers demonstrated with a quadcopter drone. To enable the drone to locate its own course, the researchers carried a camera module on the drone.
First, the demonstration is performed with the flight trajectory set. The flight trajectory set by the researchers is represented by a red line, the flight trajectory planned by the model in real time is represented by a white line, and the optimal flight trajectory finally selected by the drone is represented by a blue line. According to the simulation results, the UAV can basically fly according to the specified route.
The researchers also performed demonstrations with square and circular flight trajectories, and compared the meta-learning algorithm model with other models. The results showed that the meta-learning algorithm model made fewer errors in the route.
The researchers also noted that as the meta-learning algorithm continues to adapt, the drone’s flight performance continues to optimize.
In several practical application scenarios simulated by the researchers, the UAV also completed the task well.
1. Avoid obstacles
2. Picking up, transporting and unloading goods
3. Use the baton to plan the flight route in real time
4. Track the target flight
Conclusion: Plans to further improve model autonomy
Using a model-based meta-reinforcement learning approach, researchers at Facebook and UC Berkeley have effectively improved the ability of drones to fly with payloads.
The researchers said research will continue to enable drones to perform more complex payload transport tasks. For now, the model also requires researchers to specify when the payload is picked up and put down, according to the paper. Next, the researchers plan to develop an algorithm that would allow the model to autonomously decide when to load and unload.
The Links: 3BSC610064R1 3BSE018135R1 IGBT