Probability Theory @KSoM25

Syllabus

Evaluation Plan

• End-Semester Examination: 50 marks

• Mid-Semester Examination: 25 marks

• Quizzes: 2 quizzes, 5 marks each (total 10 marks)

• Assignments: 10–12 assignments, total 15 marks

Assignments

To encourage consistent practice, you are expected to solve one question each day from the reference text A First Course in Probability by Sheldon Ross. The specific questions to be solved will be discussed in class. In addition, each of you is required to prepare well-written notes that thoroughly cover everything discussed in three lectures and one tutorial.

• Submission Deadline: Every Wednesday, 12:00 PM.

• Format: Each assignment carries 10 marks and includes 5–6 questions, except the note-preparation assignment, which carries 20 marks.

• Grading: The final assignment score will be based on the average of all assignment marks. (Percentage of marks obtained × 0.15)

Late Submission Policy

• Submission after Wednesday 12:00 PM: deduction of 5 marks for that assignment.

• One day late submission: deduction of 10 marks for that assignment.

• Two or more days late submission: deduction of 2 marks from the final average of assignments.

Recommendation

To avoid penalties and ensure timely evaluation, it is strongly recommended that assignments be submitted by Monday of the same week. 

Office Hours

Tuesdays: 4-5 PM and Fridays: 12-1 PM.

Topics covered in

• Lecture 01: Why we study probability theory, what a random experiment is, a little bit of combinatorics.

• Lecture 02: Two combinatorial proofs (including one of the Binomial Theorem), number of integer solutions to n_1 + n_2 + ... + n_r = n, introduction to probability theory, probability of an event, observations about probabilities of related events, examples of discrete probability spaces. 

• Lecture 03: Probabilies for events of some experiments, probability as a "continuous" set function, inclusion-exclusion formula and related example.

• Lecture 04: A combinatorics question, the matching problem, an example motivating Bonferroni’s inequalities-followed by a proof of these inequalities, inclusion-exclusion formula to compute the cardinalities of union of finitely may sets, introduction to conditional probability with examples.

• Lecture 05: Examples related to conditional probability, the multiplication rule, total probabiity rule, Bayes' rule, applications of these rules, independence of events with examples, definition of trials and independent subexperiments with examples.

• Lecture 06: Conditional probability and conditional independence with examples, random variables, discrete random variables, probability mass function, examples based on these topics.

• Lecture 07: CDF of a random variable and its properties, CDF and the fact that a CDF may not be equal to CDF of a random variable, expectation of a random variable.

• Lecture 08: Formula for expecation of X in terms of pmf of X, expecation of functions of X, expecation is a linear function, definitions of variance and standard deviation, Bernoulli and Binomial random variables with examples, expectation of binomial random variable.

• Lecture 09: Variance and CDF of Binomial random variable; Poisson random variable with example, expectation, variance, CDF and binomial approximation (Poisson limits); geometric random variable with exampe, expecation and variance; negative random variable with example and expectation.

• Lecture 10: Variance of negative random variable with example; hypergeometric random variable with example; expectation and variance of hypergeometric random variable using linearity of expectation; definition of continuous random variable and probability density function.

• Lecture 11: Motivation for the name pdf of a continuous random variable X, definition of CDF of X with emphasis on how it may be used to find pmf of X, example related to continuous random variable, expectation of positive function of a continuous random variabe, uniform random variable, its expectation, variance and related example; normal random variable N along with the fact that cN+b is also a normal RV for non-zero c, using standard normal random variable (SNRV) to compute probabilities.

• Lecture 12: Expectation and variance of (SNRV); The DeMoivre-Laplace limit theorem and its applications; exponential random variable, its pdf, CDF, expectation, variance, related example and the fact that exponential random variable is the only memoryless random variable; under some assumptions involving λ>0, t >0 and independence, the number of events occuring in an interval of length t is Poisson random variable with parameter λt.

• Lecture 13: Gamma random variable and the proof that under certain assumptions the distribution of Gamma random variable (with α = n) is the same as the distribution of time until n events occur; Γ(α+1) = Γ(α), expectation and variance of Gamma random variable; where do we use Weibull random variable, Standard Cauchy random variable with example; Beta distribution, which random phenomenon does it model and relation between beta and Gamma functions.

• Lecture 14: Expectation and variance of Beta distribution; pdf of a function of random variable with examples; what is simulation, why is it needed and why it works; how to generate a random permutation; the inverse transformation method for drawing random numbers from discrete and continuous random variables, related examples.

• Lecture 15: Jointly continuous random variables, joint and marginal CDF/pdf/pmf of random variables, independence of (jointly continuous) random variables.

• Lecture 16: CDF and pdf of sums of independent random variables with examples, discrete conditional distribution.

• Lecture 17: Recapitulation of the concepts covered in lecture 16; If X,Y and Z are three independent discrete random variables then Z is independent of X+Y; Discrete, continuous and other conditional distributions as tools for conditioning over events with zero probability.

• Lecture 18: Change of variables, expectations of sums of random variable with applications.

• Lecture 19: Number of distinct subsets of k events, introduction to covariance; sample expectation and variance.

• Lecture 20: Markov's inequality, Correlation coefficient and Cauchy-Schwarts inequality for covariance and expectation, Conditional expectations.

• Lecture 21: Computing expectations and probabilities by conditioning, moment generating function and characteristic function, Levy's inversion theorem.

• Lecture 22: Joint moment generating function.

• Lecture 23: Chebyshev's inequality, weak and strong laws of large numbers, central limit theorem, Chernoff bounds, Poisson process.

• Lecture 24: Markov chain, random walks.

References

• Sheldon Ross, A first course in probability, ninth ed., Macmillan Co., New York; Collier Macmillan Ltd., London, 1984.

• William Feller, An introduction to probability theory and its applications. Vol. I, Third edition, John Wiley & Sons, Inc., New York-London-Sydney, 1968.

• William Feller, An introduction to probability theory and its applications. Vol. II, Second edition, John Wiley & Sons, Inc., New York-London-Sydney, 1971.