[TOC]

Fundamental principles of probability

Random experiment

An observation or measurement process with multiple but uncertain outcomes.

一个具有多个但不确定结果的观察或测量过程。

投掷一枚色子就是一个随机试验，可以重复，但是有不确定的结果。

Sample Space

A set containing all possible outcomes of an experiment.

Eg. the sample space of a rolling a single six-sided die is {1,2…, 6}.

Events

Subsets of sample space,i.e, a set of outcomes and may contain one or more of the values in the sample space, or it may even contain no elements.

事件包含随机变量和结果。

Event Space

The event space consists of all combinations of outcomes.

当进行一次掷骰子的随机试验时，样本空间是 {1, 2, 3, 4, 5, 6}，因为掷骰子可能出现的结果是这些数字。如果我们关注的事件是骰子的结果是偶数，那么事件空间就是 {2, 4, 6}，这是样本空间的一个子集。

另一个例子是扑克牌抽牌的随机试验。样本空间是一副扑克牌中的所有牌，总共有52张。如果我们关注的事件是抽出一张红心牌，那么事件空间就是红心牌的集合，包括 {红心A, 红心2, 红心3, …, 红心K} 这些牌。

在这些例子中，样本空间是所有可能结果的集合，而事件空间是与特定事件相关的结果的子集。

Venn Diagrams

Mutually exclusive events【互斥事件】

Events that cannot both happen at the same time.

Axioms【公理】 of Probability

Any event A in the event space has P(A)≥0 or(Pr(A)≥0).

The probability of all events in sample space is 1.

If events $A_1$ and $A_2$ are mutually exclusive, $P(A_1UA_2)=P(A_1)+P(A_2)$.

Extensions

The probability of an event or its complement must be 1
$$
\mathrm{P}\left(\mathrm{A} \cup \mathrm{A}^{\mathrm{C}}\right)=\mathrm{P}(\mathrm{A})+\mathrm{P}\left(\mathrm{A}^{\mathrm{C}}\right)=1
$$
The probability of the union of any two sets can be decomposed into【加法法则】:

$$
P(A \cup B)=P(A)+P(B)-P(A \cap B)
$$

$$
P(A+ B)=P(A)+P(B)-P(AB)
$$

Conditional probability

掌握定义和表达形式。

Unconditional Probability【非条件概率或边际概率】 (Marginal Probability)

The probability of an event without any restrictions (or lacking any prior information), commonly known as $\mathrm{P}(\mathrm{A})$.

Conditional Probability

The probability on condition that another event occurs first. The conditional probability of Event $B$, conditional on Event $A$, is given by
$$
\mathrm{P}(\mathrm{B} \mid \mathrm{A})=\frac{\mathrm{P}(\mathrm{A} \cap \mathrm{B})}{\mathrm{P}(\mathrm{A})} ; \mathrm{P}(\mathrm{A})>0
$$

Joint probability【联合概率】

$\mathrm{P}(\mathrm{A} \cap \mathrm{B})$ is the joint probability, which means the probability that two events occur simultaneously.

Total Probability Formula【非条件概率】

If an event $\mathrm{A}$ must result in one of the mutually exclusive events $A_1, A_2$, $A_3, \ldots \ldots, A_n$, then
$$
P(A)=P\left(A_1\right) P\left(A \mid A_1\right)+P\left(A_2\right) P\left(A \mid A_2\right)+\ldots+P\left(A_n\right) P\left(A \mid A_n\right)
$$

(1) $A_i \cap A_j=\emptyset(i \neq j)$

(2) $\bigcup_{i=1}^n A_i=\Omega$

Independence

Independent events(or unconditional independent events):

If the event $(B)$ is not influenced by whether the other event(A) occurs, then we say those events are independent, otherwise they are dependent.

独立是一件事的发生不受另一件事的影响；互斥是一件事情会影响另外一件事，使得另外一件事情不发生。
$$
P(A \cap B)=P(A) \times P(B)
$$

Conditional independence

Like probability, independence can be redefined to hold conditional on another event $©$, two events $A$ and $B$ are conditionally independent if :

在一件事情发生的情况下，A，B互不影响。

$$
\mathrm{P}(\mathrm{A} \cap \mathrm{B} \mid \mathrm{C})=\mathrm{P}(\mathrm{A} \mid \mathrm{C}) \times \mathrm{P}(\mathrm{B} \mid \mathrm{C})
$$

Note that two types of independence-unconditional and conditional—do not imply each other.
- Events can be both unconditionally dependent and conditionally independent.
- Events can be independent, yet conditional on another event they may be dependent.

条件独立和独立没有必然联系。

Bayes’ rule【计算条件概率】

Specifically, we are often interested in the probability of an event happening only if another event happens first.

Bayes’ rule provides a method to construct conditional probabilities using other probability measures.

It is both a simple application of the definition of conditional probability and an extremely important tool.

P(A):prior probability,先验概率,一般是通过经验给出。

P(A|B):posterior prability，后验概率,在知道B后做A的概率

P(B|A)/P(B)：information adjusted factor，信息调整因子；如果IAF等于1，则说明信息B无效。信息调整因子反映了信息的质量，越远离1，说明信息质量越高。