Advancing Over-the-Air Federated Learning through Deep Reinforcement Learning in UAV-Assisted Networks with Movable Antennas

Document Type : Internet of Thing (IoT)-Yaghmaee

Authors

1 Department of Electric and Computer Engineering, Ferdowsi University of Mashhad, Mashhad, Iran.

2 Department of Electric and Computer Engineering, Laval University, Quebec City, Canada,

Abstract

This paper investigates the deployment of over-the-air federated learning (OTA-FL), leveraging the dynamic repositioning and line-of-sight communication capabilities of unmanned aerial vehicles (UAVs) and movable antennas to enhance network efficiency. A closed-form expression is derived to quantify the optimality gap between the actual federated learning (FL) model and its theoretical ideal, accounting for the capabilities of movable antennas to show the diverse relationship between Mean Square Error (MSE) and the optimality gap. Then An MSE minimization problem is then formulated, involving the joint optimization of moveable antenna position vectors, and the beamforming vector at the UAV. This complex non-convex problem is reformulated as a Markov Decision Process (MDP) and solved using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm within the deep reinforcement learning (DRL) framework. Numerical results demonstrate that the proposed algorithm outperforms benchmarks such as Advantage Actor-Critic(A2C) and Soft Actor-Critic (SAC).
 

Keywords

Main Subjects

References

[1] M. Ahmadzadeh, S. Pakravan, G. A. Hodtani, M. Zeng, and J. Y. Chouinard. (2024, Nov.). Deep Reinforcement Learning for Robust RIS-Aided Over-the-Air Federated Learning in Cognitive Radio. Presented at 2024 IEEE Middle East Conference on Communications and Networking (MECOM). [Online]. Available: https://doi.org/10.1109/MECOM61498.2024.10881858
[2] M. Pourghasemian, H. Gacanin, and E. Perenda. (2023, Dec.). Cooperative Partial Task-Offloading for Heterogeneous Industrial Robotic MEC System Using Spectral and Energy-Efficient Federated Learning. Presented at IEEE Global Communications Conference (GLOBECOM 2023). [Online]. Available: https://doi.org/10.1109/GLOBECOM54140.2023.10436994
[3] S. Pakravan, M. Ahmadzadeh, M. Zeng, G. A. Hodtani, J. Y. Chouinard, and L. A. Rusch. (2024, Sep.). Robust Resource Allocation for Over-the-Air Computation Networks with Fluid Antenna Array. Presented at IEEE Globecom Workshops (GC Wkshps). [Online]. Available: https://doi.org/10.48550/arXiv.2504.16221
[4] M. Ahmadzadeh, S. Pakravan, G. A. Hodtani, M. Zeng, J. Y. Chouinard, and L. A. Rusch. (2024, Jul.). Enhancement of Over-the-Air Federated Learning by Using AI-based Fluid Antenna System. Presented at arXiv. [Online]. Available: https://doi.org/10.48550/arXiv.2407.03481
[5] Y. Wang, Z. Su, N. Zhang, and A. Benslimane. (2020, Aug.). Learning in the Air: Secure Federated Learning for UAV-Assisted Crowdsensing. IEEE Transactions on Network Science and Engineering. [Online]. 8(2), pp. 1055–1069. Available: https://doi.org/10.1109/TNSE.2020.3014385
[6] M. Bakhshi, S. H. Ayatollahi, and M. Akbari. (2024). Enhancing Long-Range Radar (LRR) Automotive Applications: Utilizing Metasurface Structures to Improve the Performance of K-band Longitudinal Slot Array Antennas. AEU - International Journal of Electronics and Communications. [Online]. 176, pp. 155134. Available: https://doi.org/10.1016/j.aeue.2024.155134
[7] S. Pakravan, J. Y. Chouinard, X. Li, M. Zeng, W. Hao, Q. V. Pham, and O. A. Dobre. (2023, Jul.). Physical Layer Security for NOMA Systems: Requirements, Issues, and Recommendations. IEEE Internet of Things Journal. [Online]. 10(24), pp. 21721–21737. Available:  https://doi.org/10.1109/JIOT.2023.3296319
[8] M. Bakhshi, S. H. Ayatollahi, and M. Akbari. (2024). Enhanced 2-port MIMO Antenna with Composite Two-Step Metasurface for 77 GHz Vehicle-to-Everything Applications. AEU - International Journal of Electronics and Communications. [Online]. 184, pp. 154982. Available: https://doi.org/10.1016/j.aeue.2024.155404
[9] S. Pakravan, J. Y. Chouinard, M. Zeng, X. Li, W. Hao, and O. A. Dobre. (2023, Dec.). Physical-Layer Security of RIS-Assisted Networks Over Correlated Fisher-Snedecor F Fading Channels. IEEE Internet Things Journal. [Online]. 11(9), pp. 15152–15165. Available: https://doi.org/10.1109/JIOT.2023.3343936
[10] S. Park and H. Seo. (2025, Mar.). Federated Learning Meets Fluid Antenna: Towards Robust and Scalable Edge Intelligence. Presented at arXiv. [Online]. Available: https://doi.org/10.48550/arXiv.2503.03054
[11] L.-H. Shen and Y.-H. Chiu. (2025, Apr.). RIS-Aided Fluid Antenna Array-Mounted AAV Networks.  IEEE Wireless Communications Letters. [Online]. 14(4), pp. 1049–1053. Available: https://doi.org/10.1109/LWC.2025.3531049
[12] M. Pourghasemian, M. R. Abedi, S. S. Hosseini, N. Mokari, M. R. Javan, and E. A. Jorswieck. (2022). AI-Based Mobility-Aware Energy Efficient Resource Allocation and Trajectory Design for NFV Enabled Aerial Networks. IEEE Transactions on Green Communications and Networking. [Online]. 7(1), pp. 281–297. Available: https://doi.org/10.1109/TGCN.2022.3186911
[13] M. Pourghasemian, H. Gacanin, and E. Perenda. (2023, Dec.). Cooperative Partial Task-Offloading for Heterogeneous Industrial Robotic MEC System Using Spectral and Energy-Efficient Federated Learning. Presented at GLOBECOM 2023 – IEEE Global Communications Conference. [Online]. Available: https://doi.org/10.1109/GLOBECOM54140.2023.10436994
[14] S. Sheikhzadeh, M. Pourghasemian, M. R. Javan, N. Mokari, and E. A. Jorswieck. (2021, Dec.). AI-Based Secure NOMA and Cognitive Radio-Enabled Green Communications: Channel State Information and Battery Value Uncertainties. IEEE Transactions on Green Communications and Networking. [Online]. 6(2), pp. 1037–1054. Available: https://doi.org/10.1109/TGCN.2021.3135479
[15] J. Yao and N. Ansari. (2021, Dec.). Secure Federated Learning by Power Control for Internet of Drones. IEEE Transactions on Cognitive Communications and Networking. [Online]. 7(4), pp. 1021–1031. Available: https://doi.org/10.1109/TCCN.2021.3076167
[16] J. Yao and N. Ansari. (2021, Dec.). Secure Federated Learning by Power Control for Internet of Drones. IEEE Transactions on Cognitive Communications and Networking. [Online]. 7(4), pp. 1021–1031. Available: https://doi.org/10.1109/TCCN.2021.3076167
[17] Q. V. Pham, M. Le, T. Huynh-The, Z. Han, and W. J. Hwang. (2022, May). Energy-Efficient Federated Learning Over UAV-Enabled Wireless Powered Communications. IEEE Transactions on Vehicular Technology. [Online]. 71(5), pp. 4977–4990. Available: https://doi.org/10.1109/TVT.2022.3150004
[18] Y. Yu, X. Liu, Z. Liu, and T. S. Durrani. (2023, Oct.). Joint Trajectory and Resource Optimization for RIS Assisted UAV Cognitive Radio. IEEE Transactions on Vehicular Technology. [Online]. 72(10), pp. 13643–13648. Available: https://doi.org/10.1109/TVT.2023.3270313
[19] R. Liu, K. Guo, K. An, F. Zhou, Y. Wu, Y. Huang, and G. Zheng. (2023). Resource Allocation for NOMA-Enabled Cognitive Satellite–UAV–Terrestrial Networks with Imperfect CSI. IEEE Transactions on Cognitive Communications and Networking. [Online]. 9(4), pp. 963–976. Available: https://doi.org/10.1109/TCCN.2023.3261311
[20] M. Ahmadzadeh, S. Pakravan, and G. A. Hodtani. (2024, Dec.). Movable Antenna Design for UAV-Aided Federated Learning via Deep Reinforcement Learning. Presented at 2024 15th International Conference on Information and Knowledge Technology (IKT). [Online]. Available: https://doi.org/10.1109/IKT65497.2024.10892658
[21] M. Ahmadzadeh, S. Pakravan, and G. A. Hodtani. (2024, Dec.). Movable Antenna Design for UAV-Aided Federated Learning via Deep Reinforcement Learning. In Proceedings of the 15th International Conference on Information and Knowledge Technology (IKT), pp. 91–95.
[22] X. Cao, G. Zhu, J. Xu, and S. Cui. (2022, May). Transmission Power Control for Over-the-Air Federated Averaging at Network Edge. IEEE Journal on Selected Areas in Communications. [Online]. 40(5), pp. 1571–1586. Available: https://doi.org/10.1109/JSAC.2022.3143217
Send comment about this article
CAPTCHA Image

Files

Share

How to cite

Statistics