11.1 Mathematical expectation; general random variables  (Page 4/7)

Probability as expectation

Probability may be expressed entirely in terms of expectation.

• By properties (E1) and positivity (E3a) , $P\left(A\right)=E\left[I_A\right]\ge 0$ .
• As a special case of (E1) , we have $P\left(\Omega \right)=E\left[I_{\Omega }\right]=1$
• By the countable sums property (E8) ,
  $$A=\underset{i}{\overset{}{\bigvee }}{A}_i\text{implies}P\left(A\right)=E\left[I_A\right]=E\left[\sum _i{I}_{A_i}\right]=\sum _{i}E\left[I_{A_i}\right]=\sum _{i}P\left(A_i\right)$$

Thus, the three defining properties for a probability measure are satisfied.

An indicator function pattern

Suppose X is a real random variable and $E=X^{-1}\left(M\right)=\{\omega :X\left(\omega \right)\in M\}$ . Then

To see this, note that $X\left(\omega \right)\in M$ iff $\omega \in E$ , so that $I_E\left(\omega \right)=1$ iff $I_M\left(X\left(\omega \right)\right)=1$ .

Similarly, if $E=X^{-1}\left(M\right)\cap Y^{-1}\left(N\right)$ , then $I_E=I_M\left(X\right)I_N\left(Y\right)$ . We thus have, by (E1) ,

Alternate interpretation of the mean value

INTERPRETATION. If we approximate the random variable X by a constant c , then for any ω the error of approximation is $X\left(\omega \right)-c$ . The probability weighted average of the square of the error (often called the mean squared error ) is ${\displaystyle E\left[{(X-c)}^2\right]}$ . This average squared error is smallest iff the approximating constant c is the mean value.

VERIFICATION

We expand ${(X-c)}^2$ and apply linearity to obtain

The last expression is a quadratic in c (since $E\left[X^2\right]$ and $E\left[X\right]$ are constants). The usual calculus treatment shows the expression has a minimum for $c=E\left[X\right]$ . Substitution of this value for c shows the expression reduces to $E\left[X^2\right]-E^2\left[X\right]$ .

Jensen's inequality

If X is a real random variable and g is a convex function on an interval I which includes the range of X , then

VERIFICATION

The function g is convex on I iff for each $t_0\in I=[a,b]$ there is a number $\lambda \left(t_0\right)$ such that

This means there is a line through $(t_0,g\left(t_0\right))$ such that the graph of g lies on or above it. If $a\le X\le b$ , then by monotonicity $E\left[a\right]=a\le E\left[X\right]\le E\left[b\right]=b$ (this is the mean value property (E11) ). We may choose $t_0=E\left[X\right]\in I$ . If we designate the constant $\lambda \left(E\right[X\left]\right)$ by c , we have

Recalling that $E\left[X\right]$ is a constant, we take expectation of both sides, using linearity and monotonicity, to get