Prerequisite: Background in Probability and Stochastic Processes and interest in System Modeling; Markov Chains and regenerative processes have been extensively used in modeling a wide variety of systems and phenomena. Likewise, many systems can be modeled as queueing systems with some aspect of the queue governed by a random process. Obvious examples of such systems occur in telecommunication systems, manufacturing systems, and computer systems. This course is aimed at teaching system modeling using Markov chains with special emphasis on developing queueing models. The course contents are as follows; Introduction: Review of basic probability, properties of nonnegative random variables, laws of large numbers, and the Central Limit Theorem; Renewal Processes: Basic definitions, recurrence times, rewards and renewal reward theorem, point processes, Poisson process, Wald’s equation, Blackwell’s theorem; Discrete-time Markov chains: definitions and properties, matrix representation, Perron-Frobenius theory; Continuous-time Markov chains: basic definitions, Q-matrix, birth-death processes, quasi birth-death processes; Embedded Markov processes, semi Markov processes, reversible Markov chains; Random walks; Fundamental queueing results: Little’s theorem, invariance of the mean delay, Conservation law; Markovian queues: Jackson and BCMP networks, numerical Algorithms;
M/G/1 & G/M/1 queues and G/G/1 queues; Advanced queueing models: priority, vacation and retrials in queues.