Pre-requisites: EE 325 or EE 601 or an equivalent basic course on probability and random processes; Review of probability theory, Stochastic approximation algorithms: stability and convergence, asynchronous implementations, two-time scale schemes, examples from electrical engineering, Markov chain Monte Carlo: variance reduction, simulated annealing, Markov decision processes: stochastic dynamic programming, computational schemes, state and parameter estimation, control under partial observations, adaptive control, learning algorithms.