Pre-requisites : EE 325 or EE 601 or an equivalent basic course on probability and random processes.;Review of probability theory, Controlled Markov chains: dynamic programming and computational schemes, Linear stochastic control: LQG problem, Kalman filter and separation principle, Introduction to stochastic differential equations and continuous time stochastic, control: Hamilton-Jacobi-Bellman equation, nonlinear filtering.
J. Spall, Introduction to Stochastic Search and Optimization, Wiley-Interscience, New York, 2003.
Sheldon Ross, Introduction to Stochastic Dynamic Programming, Academic Press, New York, 1995.
S. Asmussen and P. W. Glynn, Stochastic Simulation, Springer Verlag, New York, 2007.