Probability Distributions – A Detailed Guide
Probability distributions describe how probabilities are assigned to different possible values of a random variable. They are fundamental in statistics, machine learning, data science, and real-world applications like finance, medicine, and artificial intelligence.
This guide covers:
- Introduction to Probability Distributions
- Types of Probability Distributions
- Key Discrete Probability Distributions
- Binomial Distribution
- Poisson Distribution
- Key Continuous Probability Distributions
- Normal (Gaussian) Distribution
- Exponential Distribution
- Comparison of Distributions
- Applications in Machine Learning & Data Science
- Conclusion
1. Introduction to Probability Distributions
A probability distribution describes the likelihood of different outcomes of a random variable. It can be classified into discrete and continuous distributions.
- Discrete Distributions → Used when the variable takes finite or countable values (e.g., number of heads in coin flips).
- Continuous Distributions → Used when the variable can take infinite values within a range (e.g., height, weight).
2. Types of Probability Distributions
2.1 Discrete vs. Continuous Probability Distributions
| Feature | Discrete Distribution | Continuous Distribution |
|---|---|---|
| Definition | Deals with countable values | Deals with infinite values in a range |
| Example | Number of heads in coin tosses | Heights of people in a population |
| Probability Calculation | P(X=x)P(X = x) | P(a≤X≤b)P(a \leq X \leq b) |
| Key Distributions | Binomial, Poisson | Normal, Exponential |
3. Key Discrete Probability Distributions
3.1 Binomial Distribution
Definition:
The Binomial Distribution models the probability of getting exactly kk successes in nn independent trials, where each trial has two possible outcomes (success or failure). P(X=k)=(nk)pk(1−p)n−kP(X = k) = \binom{n}{k} p^k (1-p)^{n-k}
where:
- nn = total number of trials
- kk = number of successful outcomes
- pp = probability of success in each trial
- (nk)=n!k!(n−k)!\binom{n}{k} = \frac{n!}{k!(n-k)!} = number of ways to choose kk successes from nn trials
Example: Coin Toss
- If we flip a fair coin 10 times, what is the probability of getting exactly 4 heads?
- n=10n = 10, k=4k = 4, p=0.5p = 0.5
- P(X=4)=(104)(0.5)4(0.5)6P(X = 4) = \binom{10}{4} (0.5)^4 (0.5)^6
- P(X=4)=10!4!(10−4)!(0.5)10P(X = 4) = \frac{10!}{4!(10-4)!} (0.5)^{10}
- P(X=4)≈0.205P(X = 4) \approx 0.205
- So there is 20.5% probability of getting exactly 4 heads.
Characteristics:
- Used when trials are independent.
- Each trial has only two possible outcomes (success/failure).
- Probability remains constant across trials.
3.2 Poisson Distribution
Definition:
The Poisson Distribution models the probability of a given number of events occurring in a fixed interval of time or space, assuming events occur at a constant rate and independently. P(X=k)=e−λλkk!P(X = k) = \frac{e^{-\lambda} \lambda^k}{k!}
where:
- λ\lambda = average number of events in an interval
- kk = number of occurrences
- ee = Euler’s number (≈2.718\approx 2.718)
Example: Call Center
- A call center receives an average of 5 calls per hour. What is the probability of receiving exactly 3 calls in an hour?
- λ=5\lambda = 5, k=3k = 3
- P(X=3)=e−5⋅533!P(X = 3) = \frac{e^{-5} \cdot 5^3}{3!}
- P(X=3)≈0.1404P(X = 3) \approx 0.1404
- So there is a 14% probability of getting exactly 3 calls in an hour.
Characteristics:
- Used for counting events occurring in a fixed time/space.
- Events are independent.
- Events occur at a constant average rate.
4. Key Continuous Probability Distributions
4.1 Normal (Gaussian) Distribution
Definition:
The Normal Distribution (or Gaussian Distribution) is a bell-shaped curve used to model natural variations in data (e.g., heights, IQ scores, measurement errors). f(x)=1σ2πe−(x−μ)22σ2f(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{(x – \mu)^2}{2\sigma^2}}
where:
- μ\mu = mean (center of the distribution)
- σ\sigma = standard deviation (spread of the distribution)
Example: IQ Scores
- If IQ scores follow a normal distribution with μ=100\mu = 100, σ=15\sigma = 15, what is the probability of a person having an IQ between 85 and 115?
- Convert values into Z-scores: Z=X−μσZ = \frac{X – \mu}{\sigma}
- Z(85)=85−10015=−1Z(85) = \frac{85 – 100}{15} = -1
- Z(115)=115−10015=1Z(115) = \frac{115 – 100}{15} = 1
- Looking up a Z-table, the probability of −1≤Z≤1-1 \leq Z \leq 1 is 68%.
Characteristics:
- Symmetric and bell-shaped.
- Used for real-world continuous data (e.g., stock prices, heights).
- Follows Empirical Rule:
- 68% of data lies within 1 standard deviation.
- 95% of data lies within 2 standard deviations.
- 99.7% of data lies within 3 standard deviations.
4.2 Exponential Distribution
Definition:
The Exponential Distribution models the time between events in a Poisson process. f(x)=λe−λx,x≥0f(x) = \lambda e^{-\lambda x}, \quad x \geq 0
where:
- λ\lambda = rate parameter
- xx = time until next event
Example: Waiting Time in a Bus Stop
- If buses arrive every 10 minutes on average, what is the probability a bus arrives within 5 minutes?
- λ=1/10=0.1\lambda = 1/10 = 0.1
- P(X≤5)=1−e−0.1×5P(X \leq 5) = 1 – e^{-0.1 \times 5}
- P(X≤5)≈0.393P(X \leq 5) \approx 0.393
- So there is 39.3% probability a bus arrives within 5 minutes.
5. Applications in Machine Learning & Data Science
| Distribution | Application |
|---|---|
| Binomial | Predicting success/failure in experiments |
| Poisson | Modeling rare events (fraud detection, customer arrivals) |
| Normal | Feature scaling, anomaly detection |
| Exponential | Predicting waiting times, survival analysis |