Stochastic Process

Basic Concepts

Random Processes as Random Functions

PDFs and CDFs

Mean and Correlation Functions

Since random processes are collections of random variables, you already possess the theoretical knowledge necessary to analyze random processes. From now on, we would like to discuss methods and tools that are useful in studying random processes. Remember that expectation and variance were among the important statistics that we considered for random variables. Here, we would like to extend those concepts to random processes.

Mean Function of a Random Process

Autocorrelation and Autocovariance

Multiple Random Processes

Independent Random Processes

Stationary Processes

Weak-Sense Stationary Processes

Jointly Wide-Sense Stationary Processes

Cyclostationary Processes

Derivatives and Integrals of Random Processes

Gaussian Random Processes

Processing of Random Signals

Now, we will study what happens when a WSS random signal passes through a linear time-invariant (LTI) system. Such scenarios are encountered in many real-life systems, specifically in communications and signal processing. A main result of this section is that if the input to an LTI system is a WSS process, then the output is also a WSS process. Moreover, the input and output are jointly WSS. To better understand these systems, we will discuss the study of random signals in the frequency domain.

Power Spectral Density

Cross Spectral Density

Linear Time-Invariant (LTI) Systems with Random Inputs

Linear Time-Invariant (LTI) Systems
LTI Systems with Random Inputs

Power in a Frequency Band

Gaussian Processes through LTI Systems

White Noise

Counting Processes

We discussed a general theory of random processes. In this chapter, we will focus on some specific random processes that are used frequently in applications. More specifically, we will discuss the Poisson process, Markov chains, and Brownian Motion (the Wiener process).

Basic Concepts of the Poisson Process

Merging and Splitting Poisson Processes

Nonhomogeneous Poisson Processes

Discrete-Time Markov Chains

State Transition Matrix and Diagram

State Transition Diagram

Probability Distributions

Classification of States

Periodicity

Using the Law of Total Probability with Recursion

In general, a finite Markov chain might have several transient as well as several recurrent classes. As n increases, the chain will get absorbed in one of the recurrent classes and it will stay there forever. We can use the above procedure to find the probability that the chain will get absorbed in each of the recurrent classes. In particular, we can replace each recurrent class with one absorbing state. Then, the resulting chain consists of only transient and absorbing states. We can then follow the above procedure to find absorption probabilities.

Mean Hitting Times

Mean Return Times

Stationary and Limiting Distributions

Finite Markov Chains

Here, we consider Markov chains with a finite number of states. In general, a finite Markov chain can consist of several transient as well as recurrent states. As n becomes large the chain will enter a recurrent class and it will stay there forever. Therefore, when studying long-run behaviors we focus only on the recurrent classes.

If a finite Markov chain has more than one recurrent class, then the chain will get absorbed in one of the recurrent classes. Thus, the first question is: in which recurrent class does the chain get absorbed? We have already seen how to address this when we discussed absorption probabilities Thus, we can limit our attention to the case where our Markov chain consists of one recurrent class. In other words, we have an irreducible Markov chain. Note that as we showed in Example 11.7, in any finite Markov chain, there is at least one recurrent class. Therefore, in finite irreducible chains, all states are recurrent.