Federated reinforcement learning (FRL) enables multiple agents to learn collaboratively without sharing local data, but it faces key challenges, including multi-agent speedup and high communication costs. This talk focuses on on-policy FRL, where agents collect data online and adapt their policies dynamically. I will present our proposed Federated Q-Learning algorithms with multiple novel designs and rigorous theoretical guarantees. This talk will also explain technical novelty on addressing the technical difficulties unique to on-policy FRL and show the further improved performance under Markov Decision Processes with a benign structure. To the best of our knowledge, our proposed methods are the first in the literature to achieve multi-agent speedup and logarithmic communication complexity for on-policy FRL.

Biography of the speaker: Lingzhou Xue is currently a Professor of Statistics at Penn State University. He received his B.Sc. in Statistics from Peking University in 2008 and his Ph.D. in Statistics from the University of Minnesota in 2012. He was a postdoctoral research associate at Princeton University from 2012-2013. His research interests include high-dimensional statistics, nonparametric statistics, statistical and machine learning, large-scale optimization, and statistical modeling in biomedical, environmental, and social sciences. He became an Elected Fellow of the Institute of Mathematical Statistics (IMS) in 2024 and an Elected Fellow of the American Statistical Association (ASA) in 2023.