0.1 Detection performance criteria  (Page 2/3)

From a grander viewpoint, these expressions represent an achievable lower bound on performance (as assessed by the probability oferror). Furthermore, this probability will be non-zero if the conditional densities overlap over some range of values of $r$ , such as occurred in the previous example. Within regions of overlap, the observed values are ambiguous: eithermodel is consistent with the observations. Our "optimum" decision rule operates in such regions by selecting that modelwhich is most likely (has the highest probability) of generating the measured data.

Neyman-pearson criterion

Situations occur frequently where assigning or measuring the a priori probabilities ${\pi }_i$ is unreasonable. For example, just what is the a priori probability of a supernova occurring in any particular region of the sky? We clearlyneed a model evaluation procedure that can function without a priori probabilities. This kind of test results when the so-called Neyman-Pearson criterion is used to derive the decision rule.

Using nomenclature from radar, where model ${ℳ}_1$ represents the presence of a target and ${ℳ}_0$ its absence, the various types of correct and incorrect decisions have the following names.

• Detection probability

  we say it's there when it is; $P_D=({ℳ}_1\text{true}, \text{say}{ℳ}_1)$

• False-alarm probability

  we say it's there when it's not; $P_F=({ℳ}_0\text{true}, \text{say}{ℳ}_1)$

• Miss probability

  we say it's not there when it is; $P_M=({ℳ}_1\text{true}, \text{say}{ℳ}_0)$

These two probabilities are related to each other in an interesting way. Expressing these quantities in terms of thedecision regions and the likelihood functions, we have $$P_F=\int p(R, {ℳ}_0, r)\,d r$$ $$P_D=\int p(R, {ℳ}_1, r)\,d r$$ As the region $Z_1$ shrinks, both of these probabilities tend toward zero; as $Z_1$ expands to engulf the entire range of observation values, they both tend toward unity. This rather directrelationship between $P_D$ and $P_F$ does not mean that they equal each other; in most cases, as $Z_1$ expands, $P_D$ increases more rapidly than $P_F$ (we had better be right more often than we are wrong!). However, the "ultimate" situation where a rule isalways right and never wrong ( $P_D=1$ , $P_F=0$ ) cannot occur when the conditional distributions overlap. Thus, to increase the detection probability we mustalso allow the false-alarm probability to increase. This behavior represents the fundamental tradeoff in detection theory .

One can attempt to impose a performance criterion that depends only on these probabilities with the consequent decision rulenot depending on the a priori probabilities. The Neyman-Pearson criterion assumes that the false-alarm probability is constrained to be less than orequal to a specified value $\alpha$ while we maximize the detection probability $P_D$ . $$\forall P_F, P_F\le \alpha \colon \max\{Z_1 , P_D\}$$ A subtlety of the solution we are about to obtain is that the underlying probability distribution functions may not becontinuous, with the consequence that $P_F$ can never equal the constraining value $\alpha$ . Furthermore, a (unlikely) possibility is that the optimum value for the false-alarm probability is somewhat lessthan the criterion value. Assume, therefore, that we rephrase the optimization problem by requiring that the false-alarmprobability equal a value $\alpha ^\prime$ that is the largest possible value less than or equal to $\alpha$ .