🚀 go-pugleaf

This is an edited version of my earlier post on the curious discussion at
http://scienceblogs.com/deltoid/2007/04/a_new_flavour_of_global_warmin.php

Briefly, a Professor of Economics and a Lecturer in Economics wrote an
article criticising the Intergovernmental Panel on Climate Change, claiming
that "with large data sets, confidence intervals have to be increased, so a
90 per cent confidence level would not then be valid -- the hypothesis is
falsified", and that an elementary statistics textbook confirms this.

Blogger Tim Lambert said this was false, challenged them to name the book,
and noted that "It is worrying that economists at Australia's universities
have such a poor understanding of basic statistics."

(BTW, IIRC, confidence intervals are when you say things like "I am 95%
confident that the average weight of ten-year-olds in this town is between
30 and 40 kg". If you weighed them all you would know the true value
(ignoring measurement error), so you could say that you were 100% confident
that it was a particular weight (say 36 kg). But if you only weigh a random
sample, you can use a formula to calculate the range that the average is
likely to be in. Obviously, the larger the sample, the _narrower_ that
interval will be, or the _higher_ the confidence you can have that the true
value is within the interval.

But the economists say above that "with large data sets, confidence
intervals have to be increased". Strange.)

Anyway, after a bit of ducking and weaving such as saying it was in "a big
fat blue paperback with a spine that cracks easily", the economists provided
some quotes from textbooks to allegedly back up what they had written, e.g.

"For a given significance level the probability of a Type I error is held
constant whatever the sample size, and the probability of a Type II error
declines as T increases. This situation favors H0 when T is small and favors
H1 when T is large. ... the significance level would need to be a decreasing
function of sample size".

*But the quotes had been _edited_, which made them look as though they might
support the statement about falsifying the hypothesis, when in fact they did
not.*

The true quote for the example above is:

"For a given significance level the probability of a Type I error is held
constant whatever the sample size, and the probability of a Type II error
declines as T increases. This situation favors H0 when T is small and favors
H1 when T is large. In contrast, the Bayesian approach treats both errors
symmetrically and favors neither hypothesis as T increases. For the sampling
theory effect to be achieved within the Bayesian framework, we would need to
make the prior odds ration a decreasing function of sample size. Similarly,
for the Bayesian effect to be achieved within the sampling theory framework,
the significance level would need to be a decreasing function of sample
size".

Apart from the bit about significance referring to an effect of which all
mention had been deleted, note that this quote specifically says that
neither type of error _increases_ with larger data sets. It's just that they
don't necessarily decrease symmetrically.

(BTW, H0 stands for the 'null hypothesis', the claim that you are testing. A
Type I error is rejecting H0 when it is really true. A Type II error is not
rejecting H0 when it is really false. )

This is not even the worst example of what Lambert calls 'quote doctoring',
(just the shortest). In another they didn't even flag the missing text with
'...'. And there are other curiosities mentioned in my earlier post, and in
the full discussion at the link above.

You really have to wonder whether John Howard is getting advice on climate
change from people like this.