Conditional expectation, regression (Page 2/9)

Applied probability Page 2 / 9

Definition . The conditional expectation of X , given event C with positive probability, is the quantity

E [X | C] = \frac{E [I_{C} X]}{P (C)} = \frac{E [I_{C} X]}{E [I_{C}]}

Remark . The product form $E [X | C] P (C) = E [I_{C} X]$ is often useful.

A numerical example

Suppose $X \sim$ exponential $(λ)$ and $C = {1 / λ \leq X \leq 2 / λ}$ . Now $I_{C} = I_{M} (X)$ where $M = [1 / λ, 2 / λ]$ .

P (C) = P (X \geq 1 / λ) - P (X > 2 / λ) = e^{- 1} - e^{- 2} and

E [I_{C} X] = \int I_{M} (t) t λ e^{- λ t} d t = \int_{1 / λ}^{2 / λ} t λ e^{- λ t} d t = \frac{1}{λ} (2 e^{- 1} - 3 e^{- 2})

Thus

E [X | C] = \frac{2 e^{- 1} - 3 e^{- 2}}{λ (e^{- 1} - e^{- 2})} \approx \frac{1.418}{λ}

Got questions? Get instant answers now!

Conditioning by a random vector—discrete case

Suppose $X = \sum_{i = 1}^{n} t_{i} I_{A_{i}}$ and $Y = \sum_{j = 1}^{m} u_{j} I_{B_{j}}$ in canonical form. We suppose $P (A_{i}) = P (X = t_{i}) > 0$ and $P (B_{j}) = P (Y = u_{j}) > 0$ , for each permissible $i, j$ . Now

P (Y = u_{j} | X = t_{i}) = \frac{P (X = t_{i}, Y = u_{j})}{P (X = t_{i})}

We take the expectation relative to the conditional probability $P (\cdot | X = t_{i})$ to get

E [g (Y) | X = t_{i}] = \sum_{j = 1}^{m} g (u_{j}) P (Y = u_{j} | X = t_{i}) = e (t_{i})

Since we have a value for each t _i in the range of X , the function $e (\cdot)$ is defined on the range of X . Now consider any reasonable set M on the real line and determine the expectation

E [I_{M} (X) g (Y)] = \sum_{i = 1}^{n} \sum_{j = 1}^{m} I_{M} (t_{i}) g (u_{j}) P (X = t_{i}, Y = u_{j})

= \sum_{i = 1}^{n} I_{M} (t_{i}) [\sum_{j = 1}^{m}, g, (u_{j}), P, (Y = u_{j} | X = t_{i})] P (X = t_{i})

= \sum_{i = 1}^{n} I_{M} (t_{i}) e (t_{i}) P (X = t_{i}) = E [I_{M} (X) e (X)]

We have the pattern

(A) E [I_{M} (X) g (Y)] = E [I_{M} (X) e (X)] where e (t_{i}) = E [g (Y) | X = t_{i}]

for all t _i in the range of X .

We return to examine this property later. But first, consider an example to display the nature of the concept.

Basic calculations and interpretation

Suppose the pair ${X, Y}$ has the joint distribution

P (X = t_{i}, Y = u_{j})

$X =$	0	1	4	9
$Y = 2$	0.05	0.04	0.21	0.15
0	0.05	0.01	0.09	0.10
-1	0.10	0.05	0.10	0.05
$P X$	0.20	0.10	0.40	0.30

Calculate $E [Y | X = t_{i}]$ for each possible value t _i taken on by X

$E [Y | X = 0] = - 1 \frac{0.10}{0.20} + 0 \frac{0.05}{0.20} + 2 \frac{0.05}{0.20}$
$= (- 1 \cdot 0.10 + 0 \cdot 0.05 + 2 \cdot 0.05) / 0.20 = 0$
$E [Y | X = 1] = (- 1 \cdot 0.05 + 0 \cdot 0.01 + 2 \cdot 0.04) / 0.10 = 0.30$
$E [Y | X = 4] = (- 1 \cdot 0.10 + 0 \cdot 0.09 + 2 \cdot 0.21) / 0.40 = 0.80$
$E [Y | X = 9] = (- 1 \cdot 0.05 + 0 \cdot 0.10 + 2 \cdot 0.15) / 0.10 = 0.83$

The pattern of operation in each case can be described as follows:

For the i th column, multiply each value u _j by $P (X = t_{i}, Y = u_{j})$ , sum, then divide by $P (X = t_{i})$ .

The following interpretation helps visualize the conditional expectation and points to an important result in the general case.

For each t _i we use the mass distributed “above” it. This mass is distributed along a vertical line at values u _j taken on by Y . The result of the computation is to determine the center of mass for the conditional distribution above $t = t_{i}$ . As in the case of ordinary expectations, this should be the best estimate, in the mean-square sense, of Y when $X = t_{i}$ . We examine that possibility in the treatment of the regression problem in [link] .

Although the calculations are not difficult for a problem of this size, the basic pattern can be implemented simply with MATLAB, making the handling of much larger problems quite easy. Thisis particularly useful in dealing with the simple approximation to an absolutely continuous pair.

X = [0 1 4 9];             % Data for the joint distributionY = [-1 0 2];P = 0.01*[ 5  4 21 15; 5  1  9 10; 10  5 10  5];jcalc                      % Setup for calculations
Enter JOINT PROBABILITIES (as on the plane)  PEnter row matrix of VALUES of X  X
Enter row matrix of VALUES of Y  YUse array operations on matrices X, Y, PX, PY, t, u, and P
EYX = sum(u.*P)./sum(P);   % sum(P) = PX  (operation sum yields column sums)disp([X;EYX]')             % u.*P = u_j P(X = t_i, Y = u_j) for all i, j0         0
    1.0000    0.30004.0000    0.8000
    9.0000    0.8333

<< Chapter < Page Page > Chapter >>

Read also:

Get Jobilize Job Search Mobile App in your pocket Now!

100% Free Mobile Applications
Receive real-time job alerts and never miss the right job again

Source: OpenStax, Applied probability. OpenStax CNX. Aug 31, 2009 Download for free at http://cnx.org/content/col10708/1.6

Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Applied probability' conversation and receive update notifications?

Ask

	7 Physiotherapy Flashcards Set 7 By Rhodes Start Flashcards
	16 Biology 16 Gene Expression MCQ By OpenStax Start Quiz
	Vocabulary for "A Rose for Emily" By Bonnie Hurst Start Quiz
	1 Psychology Concept Test By John Gabrieli Start Test
	Chemistry Final By Briana Hamilton Start Flashcards
	22 AP Key Terms 22 The Respiratory System By OpenStax Start Key Terms
©flickr: Justin	The Last Holiday Concert Chapter 1 By Mackenzie Wilcox Start Quiz
©flickr: Hey	Anatomy Physiology Final By Joli Julianna Start Test
	15 Lec:15 Correlation Regression By Janet Forrester Start Quiz
©flickr: Aaron	Computer Literacy Exam 1 By Lakeima Roberts Start Quiz