🚀 go-pugleaf

---

A quantitative assessment of the relative merits of two discriminant 
rules, d1 and d2, can be obtained by considering the "leave-one-out 
misclassification rate."  That is, given a data set X={x1,...,xn} of n 
data points drawn from two classes, we leave each data point out in turn, 
calculating the discriminant rules from the remaining n-1 observations; 
then we classify the omitted data point separately, based on each 
discriminant.  We proceed thusly n times, counting the number of cases mj 
in which the omitted point xi is classified incorrectly by discriminant
rule dj.  Our "leave-one-out misclassification rate" is then Rj(X) = mj/n.
The discriminant rule that yields the smallest Rj is quantitatively better
*under this measure, based on this data set.*

See Silverman, B.W., (1986), *Density Estimation*, Chapman&Hall, p. 124.

Along these same lines, "likelihood crossvalidation" is a technique to 
judge the adequacy of fit of a statistical model, and is related to the
Kullback-Leibler information.  While this may be more intuitive when
applied to density-based discriminants, it is more generally applicable.

Again, see Silverman, B.W., (1986), *Density Estimation*, p. 52,
and 
Stone, M., (1974), "Cross-validatory choice and assessment of statistical 
predictions," *J. Roy. Statist. Soc. B*, Vol. 36, pp. 111-147.  
and
Hall, P., (1987), "On Kullback-Leibler loss and density estimation,"
*Ann. Stat.*, Vol. 15, No. 4, pp. 1491-1519.

cep

---