Oftentimes, when colleges design courses for incoming students as part of their academic curriculum, the vision and purpose behind this design is narrow. For example, a research institute may not be the best equipped to prepare students for industry, and vice-versa. With many topics to cover, it may be difficult to deliver clarity and do justice to each topic. As a consequence, college can occasionally feel like an unmotivated drag. Furthermore, given that the curriculum is certainly not designed to encourage much diversity, students may ask themselves where they fit in the jigsaw puzzle of fast-moving, ever innovating industry and research. This is especially true of those learning the ropes, like the undergraduates of our academic system.
One of my favorite adages is : "there's always space : we must just find it". The best creative players on football pitches are those that always have time and space because they seek it and find it superbly. The best organizers fit a myriad plans into a small time and the best negotiators and managers in any field are those that find the space for all competing opinions to fit in a cohesive framework. I too seek space : space that can be found in the lacunae left behind by our academic syllabus. In a research-oriented program, courses :
Turn abstract very early and don't talk about applications.
May appear to go on forever with the theories, as if the frontiers of research are at an undetermined horizon.
Therein lie the aforementioned lacunae.
Both the points above contribute to more than a smidgen of insularity in the field of pure mathematics. That's where I'd like to come in. We will discuss the applications of the most fundamental "aggregate" probabilistic object : the stochastic process here, with the following two aims :
Demonstrate how stochastic models are applied in various (broad) fields. All abstract models will be clearly linked to live phenomena in the real world. We will have a few modules in this course, each related to a particular field and/or task.
We will find papers that use minimal amounts of mathematics, but which are heavily cited or achieve noted results by using fairly simple tools and arguments. These papers will also use various techniques and assumptions that will underline the models' utility.
We will make sure that we are as far from an academic syllabus in our study as possible, and that we are as diverse as possible, filling every nook and cranny of application with our curiosity and willingness to discover and move forward.
First, let me introduce you to stochastic processes. Then, we'll move forward on to applications.
Stochastic processes are mathematical models of progressive randomness. That is, given a scenario where any sequence of steps (for instance, a programming algorithm, or I play a piece on the piano) occurs, stochastic processes come in because such sequence may be subject to random errors during its propagation, and therefore the final result may be error-strewn.
However, randomness is not all bad, and multiple problems often introduce randomness to "smoothen" algorithms, since they often provide averaging effects. Randomness is used in optimization and search because it provides efficiency and flexibility in scenarios where deterministic algorithms are either too rigid or too unstable. Diffusion-based generation of images relies on the ability of randomness to bring novelty to their outputs.
Some basic books on stochastic processes are included below.
Introduction to Probability Models, by Ross.
My own lectures, which are here(part 1) and here(part 2) . Make sure to watch in 1.5x speed, and for some reason my voice is very squeaky (is it like that in real life?!?)
A first course in Stochastic Processes, by Karlin and Taylor.
Essentials of Stochastic Processes, by Rick Durrett.
Stochastic Processes and their applications, by Gregory Lawler.
Discrete Stochastic Processes, by Gallager.
Probability Theory and Stochastic Processes with Applications, by Knill.
Now, let's see where we apply stochastic processes, and why you might be interested in them.
Stochastic filtering is a field of mathematics that is related to the observation of systems with inherent noise, and deals with the removal of these noises. It has applications all over the place, from traffic modelling and SoC estimation to rocket launching, navigation and robotics.
Here are some resources that students can look at to understand what stochastic filtering is about.
"Lectures on Discrete Time Filtering" by R.S.Bucy.
"Discrete Stochastic Processes and Optimal Filtering" by J-C. Bertein and R. Ceschi.
"An Introduction to the Kalman Filter" by G. Welch and G.Bishop.
"Tutorial : The Kalman filter" by T. Lacey.
Discovery of the Kalman Filter as a Practical Tool for Aerospace and Industry by J.McGee and Schmidt. Includes details on how the Kalman filter was used in the NASA Apollo mission.
How NASA used the Kalman filter in the Apollo Program, by J.Trainer.
The stochastic filtering problem: a brief historical account by D.Crisan.
Using Kalman filter algorithm for short-term traffic flow prediction in a connected vehicle environment , by Emami,Sarvi and Bagloee.
Fundamentals of stochastic filtering , A.Bain and D.Crisan.
Introduction to Kalman Filter and Its Applications , by Y.Kim and H.Bang.
Due to the participation of multiple people with their own interests and estimates, a financial market is often very difficult to predict and model. For this reason, Having a basic idea of how financial math works, and where probability comes in, is interesting, and in fact there are entire institutions and fields dedicated purely to financial modelling and prediction.
The following books are recommended reads for mathematical finance.
Stochastic Processes and the mathematics of Finance, by Jonathan Block.
Financial Mathematics : A comprehensive treatment by Campolieti and Makarov.
Stochastic Processes with applications to finance, by Kijima.
Notes on Stochastic Finance, by Privault.
Stochastic Calculus for Finance, I, by Steven E. Shreve.
Modelling extremal events for Insurance and Finance, by Embrechts, Mikosch and Kluppelberg.
Consider processes which require control, for instance landing a plane at a precise spot, or steering a recovering patient's vitals from critical to normal, or numerically locating the solution to some equation. In this process, randomness occurs either due to disturbances in the process, or in the input. The mathematical modelling of control systems along with systems encountered along the way, come under the field of stochastic optimal control. Fields such as Markov decision processes, Hidden Markov models, and natural language programming come under the field of optimal control.
The following books are recommended reads.
Stochastic Optimal control : The discrete case, by Bertsekas and Shreve.
Markov Decision Processes , by Puterman.
Discrete Time Stochastic Systems : Estimation and Control, by Söderström.
Consider multiple interacting components across a network, such as telephone lines connecting the world, or people being friends with each other on Facebook, or road/drainage networks in a city. Stochastic networks form the study of random development and behaviour of these networks across time.
This includes many sub-topics, such as drainage networks, social networks, and social balance theory for instance. Recent applications include stochastic neural networks.
The following books/articles are recommended reads for stochastic networks.
Lecture Notes on Stochastic Networks, by Kelly.
Dynamics of Drainage Under Stochastic Rainfall in River Networks, by Ramirez and Constantinescu.
Social Balance on Networks : Local Minima and Best Edge Dynamics, by Krishnendu Chatterjee et. al.
Social Balance on Networks: The Dynamics of Friendship and Enmity by Antal et. al.
Introduction to Stochastic neural networks, by Kappen.
Branching processes arise in the context of growth or decay of numbers randomly, such as radioactive decay, population growth or genetic drift across generations. They are a fascinating form of stochastic process.
The following books/articles are recommended reads for branching processes.
Branching Processes and their applications : a report, see here for the link.
Branching Processes in Biology, by Kimmel and Axelrod.
Branching Processes, by Athreya and Ney.
Applications of Stochastic Processes in Biology (Click to get the link)
Processes that tend to show self-similarity, or regenerate themselves at intermediate points, are known as renewal processes. They are heavily applied in the fields of risk and insurance, for instance, but also find applications in queueing.
Queues, as the name suggests, are mathematical models of systems that involve agents arriving, being serviced, and departing at random times. Since the arrival of agents is often regenerative in nature, renewal process find heavy use in these models.
Here are some good resources around these topics.
Stochastic Processes and Queues, a course by Prof. Parimal Parag.
The newly emerging field of generative AI is heavily driven using multiple stochastic processes to introduce novelty into the picture. It also connects to many other novel topics, for instance Bayesian networks and stochastic flows on these networks.
This is definitely a topic worth studying.
Here are some good resources around this.
Generative Adversarial Nets, by Goodfellow et. al.
Autoregressive model, Wikipedia.
Here are some miscellaneous applications of stochastic processes.
These processes typically occur when random agents, or urns, exchange objects (such as balls or money) among each other randomly in a network. They have some applications, you can pick any one over here.
Autoregressive processes are very important in the fields of climate modelling and weather prediction, since they are modelled as "how to predict the future using the past". A nice slide which contains some details of processes can be found here. By William S. Wei, this book also provides an excellent introduction to autoregressive processes.
Now come the applications that I don't think I can even discuss! For example, the drift of a fad or fashion across individuals is modelled by a stochastic process in this landmark paper. Two papers by Abhijit Banerjee which also model propagation of behaviour across humans, can be found here and here.