The course will consider Markov processes in discrete and continuous time. The theory is illustrated with examples from operation research, biology and economy.

Learning Outcomes After completed course, the students are expected to be able to:

Carry out derivations involving conditional probability distributions and conditional expectations. Define basic concepts from the theory of Markov chains and present proofs for the most important theorems. Compute probabilities of transition between states and return to the initial state after long time intervals in Markov chains. Identify classes of states in Markov chains and characterize the classes. Determine limit probabilities in Markov chains after an infinitely long period. Derive differential equations for time continuous Markov processes with a discrete state space. Solve differential equations for distributions and expectations in time continuous processes and determine corresponding limit distributions.