Fixed-point Error Bounds for Mean-payoff Markov Decision Processes

Explore optimal transport techniques for deriving finite-time error bounds in reinforcement learning for mean-payoff Markov decision processes in this 58-minute Google TechTalk. Delve into stochastic Krasnoselski—Mann fixed point iterations for nonexpansive maps, examining sufficient conditions for almost sure convergence towards fixed points. Analyze non-asymptotic error bounds and convergence rates, with a focus on martingale difference noise and its impact on variances. Investigate the case of uniformly bounded variances and its applications in Stochastic Gradient Descent for convex optimization. Gain insights from Roberto Cominetti, a professor at Universidad Adolfo Ibáñez, whose expertise spans convex analysis, game theory, and transportation network applications.

Fixed-point Error Bounds for Mean-payoff Markov Decision Processes