🚀 go-pugleaf

>On  Fri, 31 Jan 1992 Evan Cooch <ECOOCH@PENNSAS.UPENN.  EDU> writes:
>>
>> QUESTION: IF YOU CAN'T USE THE SAME TRANSFORM FOR ALL YOUR DATA,
>> SHOULD YOU TRANSFORM AT ALL?

To which Bill Knight replies:
>
>Question: If you can't use the same transform for all your data,
>SHOULD YOU USE ANOVA AT ALL?
>
>(With data that nasty, nonparametrics look good don't they?)
>

And the envelope, please.....

NO!

*****  nonparametrics don't solve the problem at all!!! *****

There seems to be a common misconception that nonparametric methods are
assumption-free methods.  Take, for example, the simple two-sample Wilcoxon
test ( = Mann-Whitney).  There are some pretty restrictive assumptions,
which include not only independence of observations (fairly standard), but
also...
EQUAL VARIANCES.  And equal skewness.  And equal kurtosis. And, well, you
get the picture.

The null hypothesis is that the X's and Y's come from the same
distribution.  The alternative [here comes the assumption] is that the X's
and Y's come from the same distribution after the Y's are all shifted by a
constant amount, say M.

Note: unequal variances, or shapes, invalidates the assumption.


Remarks.

(1)  Actually the Wilcoxon procedure is applicable under a less restrictive
set of conditions, but under those assumptions it is no longer a test of
equal medians.

(2)  Yes, I know that the underlying distribution might not have a variance
(or higher moments); I'm trying to keep things simple.

(3)  Exactly the same remarks apply to the Kruskal-Wallis test (one-way
anova based on ranks) --- except for the difference in medians, the groups
all have the *same* distribution, including identical variances (if any).

(4)  Most nonparametric methods have similar assumptions.  As a result,
nonparametric methods are not
        (a)  A free lunch
        (b)  Necessarily more applicable in practice
        (c)  Safer than doing normal statistics
        (d)  Less prone to error than other methods, and,
most importantly, they are
        (e)  NOT ASSUMPTION-FREE.

(5)  An excellent book on nonparametrics (in part because it spells out the
assumptions in considerable detail), is Nonparametric Statistical Methods,
by Myles Hollander and Douglas Wolfe (Wiley).


Bill did ask the right question:
>Question: If you can't use the same transform for all your data,
>SHOULD YOU USE ANOVA AT ALL?

Despite it's name, anova is essentially examines whether the means of the
groups are plausibly the same.  As such, it is most appropriate
CONCEPTUALLY when the groups under study are such that the main important
way in which they might differ is through a shift in the means.  If that
isn't the case, it is unlikely that anova will come close to answering your
question.  It is often (but not always) the case that transforming the data
so that the spreads are roughly comparable makes comparison of means more
readily interpretible.  Occasionally, this transformation makes the
distributions more symmetric as well, which also makes it easier to think
in terms of means.  Transformations are not a panacea but, often as not,
they do help out.

If the groups seem to have different distributional shapes, rethink the
idea of shoving the data through an anova program.  By thinking carefully,
you may realize that there is a different question (i.e., not equality of
means) that you really should be asking.

Here is a (true) example.  One indicator of whether a prescription drug has
abuse potential is whether people like the drug.  Liking means choosing the
drug more often than a placebo when repeatedly given a choice.  This and
other measures can be combined to produce what we'll call a Liking score.
Let's suppose that scores around zero are neutral, negative scores mean
dislike, and positive scores liking.  An investigator studied three drugs
and a placebo, studying many subjects in a controlled setting.  [The actual
study was blocked, so that each subject was exposed to each drug; for
exposition suppose that subjects were randomized to four treatment groups.]
 The mean scores for all four compounds were near zero.  Anova is nowhere
close to significant.  Let's all breathe a sigh of relief.  But as it turns
out, the scores for all subjects were all very close to zero for placebo
and two of the drugs, but there was a very high variance for drug number 3.
 The correct interpretation was *not* that none of these drugs had abuse
potential---the anova answer.  Rather, drug #3 elicits strong reactions in
many subjects, so that a third of subjects hate it, a third don't care, but
a third REALLY like it.  A drug that a third of the population really likes
may well be said to have abuse potential.

Note, the "statistical problem" here shows up as unequal variances.  Either
ignoring the problem or hunting up an unequal-variances version of anova
would miss the answer entirely.  Thinking about the data and what they mean
can lead to insight, and to better statistics.

May your days be merry and bright,