Advancing Over-the-Air Federated Learning through Deep Reinforcement Learning in UAV-Assisted Networks with Movable Antennas

Articles in Press

Document Type : Internet of Thing (IoT)-Yaghmaee

Authors

• mohsen ahmadzadeh ¹
• saeid pakravan ²
• Ghosheh Abed Hodtani ³

¹ Department of Electric and Computer Engineering, Ferdowsi University of Mashhad, Mashhad, Iran

² Department of Electric and Computer Engineering, Laval University, Quebec City, Canada,

³ Department of Electric and Computer Engineering, Ferdowsi University of Mashhad, Mashhad, IranAzadi Square, Mashhad, Razavi Khorasan

10.22067/cke.2025.91290.1139

Abstract

This paper investigates the deployment of over-the air federated learning (OTA-FL), leveraging the dynamic

repositioning and line-of-sight (LOS) communication capabilities of

unmanned aerial vehicles (UAVs) and movable antennas (MAs) to

enhance network efficiency. A closed-form expression is derived to

quantify the optimality gap or convergence gap between the actual federated learning

(FL) model and its theoretical ideal, accounting for the capabilities

of movable antennas to show the diverse relationship between

Mean Square Error (MSE) and the optimality gap. Then An MSE

minimization problem is then formulated, involving the joint

optimization of moveable antenna position vectors, and the

beamforming vector at the UAV. This complex non-convex

problem is reformulated as a Markov Decision Process (MDP) and

solved using the Twin Delayed Deep Deterministic Policy Gradient

(TD3) algorithm within the deep reinforcement learning (DRL)

framework. Numerical results demonstrate that the proposed

algorithm outperforms benchmarks such as Advantage ActorCritic (A2C) and Soft Actor-Critic (SAC).

Keywords

• Over-the-air federated learning
• Deep reinforcement learning
• Unmanned aerial vehicles
• Movable Antenna

Main Subjects

• Software Engineering Internet of Thing (IoT)

---