Role of Probability density function in machine learning

Introduction

Laymen explanation

Probability theory provides a consistent framework for the quantification and manipulation of uncertainty. If you are interested to know role of PDF (probability density function) tool in machine learning, then this document helps.

Technical explanation

A probability distribution is a summary of probabilities for the values of a random variable.

As a distribution, the mapping of the values of a random variable to a probability has a shape when all values of the random variable are lined up. Total area of this shape will always be 1. In below example, the shaded region area sums to 1. This value (1) came from the probability rule which tells that sum of probabilities of all values in the sample space must be 1.

Mathematical properties

• The probability density function is non-negative for all the possible values, i.e. f(x)≥ 0, for all x

• The area between the density curve and horizontal X-axis is equal to 1, i.e. ∫∞−∞f(x)dx=1

Examples of PDFs

Uniform distribution

This is simplest PDF and widely used for random number generation

Normal (Guassian) distribution

This distribution is being used in popular central limit theorem. Here, the two parameters μ and σ entirely define the shape and all other properties of the normal distribution function.

PDF's Role in data science

Probability distributions play an important role in machine learning from the distribution of input variables to the models, the distribution of errors made by models, and in the models themselves when estimating the mapping between inputs and outputs.

Commonly used PDFs in data science

Reference

https://images.app.goo.gl/xGJ4pCXtccqrfbUV8

https://www.math24.net/probability-density-function/

https://machinelearningmastery.com/what-are-probability-distributions/

https://www.cs.toronto.edu/~urtasun/courses/CSC411_Fall16/tutorial1.pdf

https://machinelearningmastery.com/continuous-probability-distributions-for-machine-learning/

https://www.kdnuggets.com/2020/02/probability-distributions-data-science.html

https://byjus.com/maths/probability-density-function/