Random experiments and events
Every probability calculation starts with a clearly defined random experiment and a well-specified sample space. Getting these right is the foundation of everything else in probability theory.
Random experiment
A random experiment is a process that produces an outcome that cannot be predicted with certainty, but whose set of possible outcomes is fully known in advance. Three conditions define it:
- The experiment can be repeated under the same conditions.
- All possible outcomes are known before the experiment is performed.
- The actual outcome of any single trial is unpredictable.
- A quality control inspector selects one unit from a production batch and classifies it as defective or non-defective. The outcome is uncertain, but the two possible results are known.
- A clinical trial enrolls a patient and records whether they respond to a treatment. Response or no response are the two possible outcomes.
- A meteorologist records the daily maximum temperature. The exact value is unknown beforehand, but it belongs to a well-defined range.
- A network engineer monitors whether a server is up or down at a given time.
⚠️ Not all uncertain processes are random experiments
A random experiment requires the set of possible outcomes to be well-defined in advance. “What will the stock market do next year?” is uncertain, but if the set of outcomes is not clearly specified (what counts as “up”? by how much?), it does not qualify as a properly defined random experiment in the probabilistic sense. Vague experiments lead to vague probabilities.
Sample space
The sample space \(\Omega\) (sometimes written \(S\)) is the set of all possible outcomes of a random experiment. Defining it correctly is the first and most important step in any probability problem.
Sample spaces can be:
- Finite: a countable set of distinct outcomes. \(\Omega = \{0, 1\}\) for a binary trial.
- Countably infinite: outcomes are countable but unlimited. \(\Omega = \{0, 1, 2, 3, \ldots\}\) for the number of customers arriving in a day.
- Continuous: outcomes form an interval. \(\Omega = [0, \infty)\) for a waiting time in minutes.
Clinical trial: a drug is tested on a patient and the number of days until recovery is recorded.
\[\Omega = \{1, 2, 3, \ldots\} \cup \{\text{no recovery}\}\]
Quality control: a batch of 5 items is inspected and the number of defectives is counted.
\[\Omega = \{0, 1, 2, 3, 4, 5\}\]
Server response time: the time in milliseconds until a server responds to a request.
\[\Omega = (0, \infty)\]
Survey response: a respondent rates satisfaction on a 5-point scale.
\[\Omega = \{1, 2, 3, 4, 5\}\]
💡 Define the sample space before computing any probability
Many probability errors come from an incorrectly defined sample space. Before writing a single probability, ask: what are all the possible outcomes? Are they mutually exclusive (no two can happen simultaneously)? Are they exhaustive (do they cover every possibility)? A good sample space is both mutually exclusive and exhaustive.
Events
An event is any subset of the sample space. An event occurs if the outcome of the experiment belongs to that subset.
- A simple event (or elementary event) contains exactly one outcome.
- A compound event contains two or more outcomes.
Quality control experiment: \(\Omega = \{0, 1, 2, 3, 4, 5\}\) defectives in a batch of 5.
- Simple event: exactly 2 defectives. \(A = \{2\}\).
- Compound event: fewer than 3 defectives. \(B = \{0, 1, 2\}\).
- Compound event: at least 1 defective. \(C = \{1, 2, 3, 4, 5\}\).
The probability of event \(B\) using classical probability (if all counts were equally likely):
\[P(B) = \frac{|B|}{|\Omega|} = \frac{3}{6} = 0.5\]
In practice, the counts are not equally likely (they follow a binomial distribution), but the event structure is the same.
Types of events
Certain and impossible events
- The certain event is \(\Omega\) itself: it always occurs because the outcome is always in the sample space. \(P(\Omega) = 1\).
- The impossible event is \(\emptyset\): it never occurs. \(P(\emptyset) = 0\).
⚠️ P(A) = 0 does not mean A is impossible
In a continuous sample space, every individual outcome has probability zero. If \(X\) is the exact response time of a server in milliseconds, \(P(X = 142.37) = 0\) even though 142.37 ms is a perfectly possible response time. The event \(\{142.37\}\) is not impossible, it just has probability zero because probabilities in continuous spaces are defined over intervals, not points.
This is a fundamental difference between discrete and continuous probability.
Mutually exclusive events
Two events \(A\) and \(B\) are mutually exclusive (or disjoint) if they cannot both occur in the same trial: \(A \cap B = \emptyset\).
A patient is classified as: no response, partial response, or full response. These three events are mutually exclusive: a patient cannot be in two categories simultaneously.
\[P(\text{no response} \cup \text{partial response}) = P(\text{no response}) + P(\text{partial response})\]
The addition simplifies because there is no overlap.
Independent events
Two events \(A\) and \(B\) are independent if the occurrence of one does not affect the probability of the other:
\[P(A \cap B) = P(A) \cdot P(B)\]
A factory has two machines on separate lines. Machine A produces a defective item with probability 0.02. Machine B produces a defective item with probability 0.03. Whether machine A produces a defect does not affect machine B.
\[P(\text{both defective}) = 0.02 \times 0.03 = 0.0006\]
\[P(\text{at least one defective}) = 1 - P(\text{neither}) = 1 - 0.98 \times 0.97 = 1 - 0.9506 = 0.0494\]
Complementary events
The complement of event \(A\), written \(\bar{A}\) or \(A^c\), is the event that \(A\) does not occur. It consists of all outcomes in \(\Omega\) not in \(A\):
\[P(\bar{A}) = 1 - P(A)\]
Exhaustive events
A set of events is exhaustive if their union covers the entire sample space: at least one of them must occur in every trial.
A customer survey classifies respondents as: Detractor, Passive, or Promoter (NPS categories). These three categories are:
- Mutually exclusive: a respondent belongs to exactly one category.
- Exhaustive: every respondent falls into one of the three.
Together they form a partition of the sample space. The probabilities must sum to 1:
\[P(\text{Detractor}) + P(\text{Passive}) + P(\text{Promoter}) = 1\]
Probability of events
For a finite sample space with equally likely outcomes:
\[P(E) = \frac{|E|}{|\Omega|}\]
where \(|E|\) is the number of outcomes in \(E\) and \(|\Omega|\) is the total number of outcomes.
For non-equally-likely outcomes, the probability of a compound event is the sum of the probabilities of its elementary events:
\[P(E) = \sum_{\omega \in E} P(\{\omega\})\]
A spam filter classifies incoming emails. Based on historical data:
| Outcome | Probability |
|---|---|
| Spam, correctly identified | 0.25 |
| Spam, missed (false negative) | 0.05 |
| Legitimate, correctly identified | 0.65 |
| Legitimate, blocked (false positive) | 0.05 |
Event \(A\) = “email is spam”: \(A = \{\text{correctly identified spam}, \text{missed spam}\}\)
\[P(A) = 0.25 + 0.05 = 0.30\]
Event \(B\) = “filter makes an error”: \(B = \{\text{missed spam}, \text{false positive}\}\)
\[P(B) = 0.05 + 0.05 = 0.10\]
