ISEN 340: Operations Research II

Free Resources 

Below are links to helpful, free resources to use during the class. I took this class with Dr. E during the Fall 2023 semester. Here is some advice for ISEN 340:

• For exam 2, it is important to know the types of states (transient, recurrent, absorbing) in a Markov chain to help understand what calculation should be made. For example, only ergodic chains have steady state probabilities (π) and only terminating chains have absorption probabilities and conditional mean 1st passage times. 

• Knowing which type of states (absorbing vs transient vs recurrent) are in the problem can further help you deduce what the word problem wants you to find.

• Also know if you need to find a probability or the number of visits. For example, mean 1st passage time finds the expected number of visits to 1st reach a state whereas hit probability calculates the probability that a state is every visited. 

Quick Guides ($12.78 each)

Quick Guide 1-3

Coming Soon!

Quick Guide 4: Higher Order & Nonstationary Markov Chains

Key points on assumptions broken with higher order chains, Nth order Markov chain definition and example, definition of nonstationary Markov chains and examples. 

Quick Guide 5: Walks and Transitions

Key points on transitional probability, walk probability computations, n-step transition probability definition and computation, introduction to matrix multiplication using Excel. 

Quick Guide 6: Visits & Sojourn Times

Key points on state probability computation with examples, computation of the expected number of visits, definition/ theory of sojourn time, and computation of mean sojourn time with an example. 

Quick Guide 7: First Passage, Return, & Steady State Probabilities

Key points on the definitions of passage time, first passage probability, return time, first return probability, recurrence probability, steady state probability theory, and step-by-step guide how to solve for the steady state probability.

Quick Guide 8: Mean First Passage Time

Key points on the definition of passage time, mean first passage time, mean recurrence time, and a step-by-step example on how to find both the mean first passage and mean recurrence time. 

Quick Guide 9: Terminating Chains

Key points on the differences between terminating and absorbing states, the breakdown of a transition matrix into its Q,R,0,& I components, explanations of expected duration, absorption probability, and hit probability.

Quick Guide 10: Conditional Mean 1st Passage Time

Key points on how to compute the conditional mean first passage time with a step-by-step example and demonstration how to solve in Excel.

Quick Guide 11: Classifying Markov Chains

Key points on the differences in the definition of recurrent and transient states, how to identify a communicating class, the indicators of ergodic vs terminating Markov chains, and a step-by-step guide on how to classify a chain as either terminating, ergodic, or nether (with examples). 

Quick Guide 12: Continuous Time Stochastic Processes

Key points on the differences between discrete and continuous time stochastic processes, continuous time transition diagrams with illustrations, the Poisson random variable, the interarrival time random variable, the random variable for the arrival of the nth event, and theory of superposition.

Quick Guide 13-18 

Coming Soon!

Quick Guide 19: Birth Death Process & Rate Sojourn Time

Key points on the linear (pure) birth process, linear growth model with external source (immigration in the chain), single server queuing system (M|M|1), multi server queuing system (M|M|S), how to calculate sojourn time with rates. 

Quick Guide 20: Embedded Markov Chains

Key points on how to calculate the embedded Markov transition probability and examples of how to create the embedded Markov transition matrix.