Stochastic Control and Communication

ABOUT THE COURSE:The legacy of control is marked by a historic misconception that stochastic control problems could be solved using deterministic controllers superimposed with optimal state estimators and conventional communication protocols. Over time, we realized that this architecture is suboptimal and co-design of communication and control offers orders of magnitude of improvements.This applies to systems such as smart cities, transportation systems, power grids, gaming and financial markets. This complex subject requires simultaneous application of control theory and communication theory. At present these elements are taught separately in courses designed either exclusively for deterministic control theory, or exclusively for communication theory. This leaves behind a gap, namely blending insights from both disciplines to arrive at approaches for networked control problems. This course seeks to fill this gap.It will begin with centralized stochastic control, and then highlight open issues that arise in decentralization and the role of communication theory. Finally it will cover communication theory and establish viewpoints from which stochastic control and communication theory can be approached simultaneously.PRE-REQUISITES: Comfort with probability.INTENDED AUDIENCE: Students, researchers and practitioners of control and automation across any discipline.INDUSTRY SUPPORT:Industries working in control and automation, decentralized multiagent systems.

Week 1: Markov decision process, finite horizon problem formulation, examples, principle of optimality, Bellman equationWeek 2:Infinite horizon problems, Optimality criteria (average cost, discounted cost), Bellman equation, optimality of Markov policiesWeek 3:Computing optimal policies, linear programming formulationWeek 4:Partially observed Markov decision processes, reduction to the information stateWeek 5:LQR problem, Kalman filterWeek 6:LQG problem, separation principle, optimality of linear policiesWeek 7:Witsenhausen counterexample. information structure,Week 8:Intrinsic model of stochastic control, LQG static teams, optimality of linear policiesWeek 9:Variants of the Witsenhausen problem, Bansal Basar problem, optimizer’s approachWeek 10:Communication and decentralized control. Canonical communication problems of source coding, channel coding and rate distortion theoryWeek 11:Shannon’s coding theoremsWeek 12:Shannon’s coding theorems and optimizer’s approach