<< Chapter < Page Chapter >> Page >

Determining a sufficient statistic directly from the definition can be a tedious process. The following result can simplify this process byallowing one to spot a sufficient statistic directly from the functional form of the density or mass function.

Fisher-neyman factorization theorem

Let f x be the density or mass function for the random vector x , parametrized by the vector . The statistic t T x is sufficient for if and only if there exist functions a x (not depending on ) and b t such that f x a x b t for all possible values of x .

In an earlier example we computed a sufficient statistic for a binary communication source (independent Bernoulli trials) from thedefinition. Using the above result, this task becomes substantially easier.

Bernoulli trials revisited

Suppose x n Bernoulli are IID, n n 1 , , N . Denote x x 1 x n . Then

f x n 1 N x n 1 1 x n k 1 N k a x b k
where k n 1 N x n , a x 1 , and b k k 1 N k . By the Fisher-Neyman factorization theorem, k is sufficient for .

The next example illustrates the appliction of the theorem to a continuous random variable.

Normal data with unknown mean

Consider a normally distributed random sample x 1 , , x N 1 , IID, where is unknown. The joint pdf of x x 1 x n is f x n 1 N f x n 1 2 N 2 -1 2 n 1 N x n 2 We would like to rewrite f x is the form of a x b t , where dim t N . At this point we require a trick-one that is commonly used when manipulating normal densities, and worthremembering. Define x 1 N n 1 N x n , the sample mean. Then

f x 1 2 N 2 -1 2 n 1 N x n x x 2 1 2 N 2 -1 2 n 1 N x n x 2 2 x n x x x 2
Now observe
n 1 N x n x x x n 1 N x n x x x x 0
so the middle term vanishes. We are left with f x 1 2 N 2 -1 2 n 1 N x n x 2 -1 2 n 1 N x 2 where a x 1 2 N 2 -1 2 n 1 N x n x 2 , b t -1 2 n 1 N x 2 , and t x . Thus, the sample mean is a one-dimensional sufficient statistic for the mean.

Proof of theorem

First, suppose t T x is sufficient for . By definition, f T x t x is independent of . Let f x t denote the joint density or mass function for ( X , T ( X ) ) . Observe f x f x t . Then

f x f x t f t x f t a x b t
where a x f t x and b t f t . We prove the reverse implication for the discrete case only. The continuous case follows a similar argument, butrequires a bit more technical work ( Scharf, pp.82 ; Kay, pp.127 ).

Suppose the probability mass function for x can be written f x a x b x where t T x . The probability mass function for t is obtained by summing f x t over all x such that T x t :

f t x T x t f x t x T x t f x x T x t a x b t
Therefore, the conditional mass function of x , given t , is
f t x f x t f t f x f t a x x T x t a x
This last expression does not depend on , so t is a sufficient statistic for . This completes the proof.

From the proof, the Fisher-Neyman factorization gives us a formula for the conditionalprobability of x given t . In the discrete case we have f t x a x x T x t a x An analogous formula holds for continuous random variables ( Scharf, pp.82 ).

Further examples

The following exercises provide additional examples where the Fisher-Neyman factorization may be used toidentify sufficient statistics.

Uniform measurements

Suppose x 1 , , x N are independent and uniformly distributed on the interval 1 2 . Find a sufficient statistic for 1 2 .

Express the likelihood f x in terms of indicator functions.

Poisson

Suppose x 1 , , x N are independent measurements of a Poisson random variable with intensity parameter : x x 0 , 1 , 2 , f x x x

Find a sufficient statistic t for .

What is the conditional probability mass function of x , given t , where x x 1 x N ?

Normal with unknown mean and variance

Consider x 1 , , x N 2 , IID, where 1 and 2 2 are both unknown. Find a sufficient statistic for 1 2 .

Use the same trick as in .

Get Jobilize Job Search Mobile App in your pocket Now!

Get it on Google Play Download on the App Store Now




Source:  OpenStax, Pdf generation problem modules. OpenStax CNX. Sep 23, 2008 Download for free at http://cnx.org/content/col10514/1.4
Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Pdf generation problem modules' conversation and receive update notifications?

Ask