🚀 go-pugleaf

On 18 mai, 15:02, "hot frog" <s...@tension.span> wrote:
> I found this discussion very interesting. Make up your own mind:        
>
> http://scienceblogs.com/deltoid/2007/04/a_new_flavour_of_global_warmi...
>
> Basically, Sinclair Davidson, a Professor of Economics, and Alex Robson, a Lecturer in Economics, wrote an article, part of which was published in the Australian Financial Review, criticising the Intergovernmental Panel on Climate Change (IPCC) and including this statement:
>
> “As an elementary textbook of statistics reminds its readers, with large data sets, confidence intervals have to be increased, so a 90 per cent confidence level would not then be valid -- the hypothesis is falsified.”
>
> Blogger Tim Lambert dissects their article and dismisses the above claim:
>
> "No, elementary textbooks of statistics do not say that. Mainly because it's not true. A larger data set means that  smaller effects will be statistically significant, so it is possibly that an effect could be statistically significant but so small that it is not practically significant. Furthermore, if a test does not achieve a specified confidence level, it does not falsify the hypothesis -- all you can say is that there is insufficient data to draw a conclusion."
>
> concluding
>
> "It is worrying that economists at Australia's universities have such a poor understanding of basic statistics."
>
> The economists respond, and some of their arguments are remarkable.
>
> Firstly, they repeatedly cite Lindley's paradox as a reason for doubting the IPCC’s conclusions, e.g.:
>
> "The IPCC claim to have a large data set, and if they do then Lindley's paradox might be a problem. Now do they have a lerge data set or don't they? If they do then Lindleys paradox is a problem, if they don't and Lindley isn't a problem then they don't have a large sample."
>
> but Wikipedia states that Lindley's paradox only applies to hypothesis testing problems in cases where
>
> " the prior distribution is the sum of a sharp peak at H0 with probability p and a broad distribution with the rest of the probability 1-p. It is a result of the prior having a sharp feature at H0 and no sharp features anywhere else."
>
> They also hammer the point that the IPCC’s Summary for Policymakers does not explain exactly how they arrived at their figure of  >90% for the assessed likelihood that most of the observed increase in globally averaged temperatures since the mid-20th century is due to the increase in anthropogenic greenhouse gas concentrations.
>
> "So the basic question we raise still remains unanswered: … where does their 90% figure come from?"
>
> "The point is though, we don't know how data the IPCC have because they don't reveal how they come to their 90% estimate."
>
> "How is the 90% value derived? If they were transparent as they are required to be, then we wouldn't be having this debate."
>
> "They have reported a number (indeed several numbers) and all I'm asking for is an explanation of where that number came from."
>
> However, when asked "whom you attempted to contact that is affiliated with the IPCC that was unwilling to answer your question(s)", the reply was:
>
> "I am unable to contact the IPCC directly because I don't have ethics approval to collect data from human subjects." !!
>
> Finally, challenged to name one elementary textbook of statistics which proves their assertion that:
>
> “As an elementary textbook of statistics reminds its readers, with large data sets, confidence intervals have to be increased, so a 90 per cent confidence level would not then be valid -- the hypothesis is falsified.”,
>
> Professor Davidson tries to dodge the question with truisms like: "I suspect all of them will have section on hypothesis testing. Most chapters include a discussion of confidence levels, Type I and Type II errors and the like", then changes the subject.
>
> This exchange follows. (I’ve extracted it from all the other debate going on in the thread):
>
> Lambert:  "If your statement about stats texts is true, it would be easy for you to provide just one example. Why can't you do this? Is it because your claim is a fabrication?"
>
> Davidson:  "Fabrication in what sense?"
>
> Lambert:  "Fabrication in the sense that elementary texts do not say that "with large data sets, confidence intervals have to be increased" but rather you and Alex Robson made that part up. It is trivial to prove that you didn't -- just cite one elementary text that says it. How about it?"
>
> Davidson:  "That's a very agressive stance you're taking Tim. You're leaving no wiggle room for yourself. I'm wondering what you're going to do when I name a book?
>
> I'll give you a clue. While some hardcover versions did come out, it's a big fat blue paperback with a spine that cracks easily."
>
> Lambert:  "Sinclair, the normal practice when citing a book is to give the author and title. I'm afraid that "big fat blue paperback" isn't a very useful way of citing."
>
> Davidson:  "Tim, the book is Judge etal. Intro to practice of econometrics (title from memory) it's out of print now. Another is by Zellner that's an intro to Baysian stats (probably of out of print too). Leamer wrote about this as well in one of his books (probably not an intro, from memory)."
>
> Lambert:  "Thanks for finally providing a proper cite, Sinclair. The UNSW library has a copy of "Introduction to the Theory and Practice of Econometrics" (2nd Edition) by Judge et al. In section 3.5 Hypothesis Testing I find (page 98):
>
> As a final note on the power of a test, note that for a given level of significance of the test, α, as the sample size increases (to infinity) the power of the test increases (to one) for every β ≠ 1. The intuition for this is based in the fact that the test is constructed around a consistent estimator. As the sample size increases, the sampling distribution of the estimator collapses about the true parameter value, and thus the probability that we would reject a false hypothesis goes to unity. Tests with this property are called consistent tests.
>
> Judge provides no support (in over a thousand pages) for Sinclair's claim:
>
> As an elementary textbook of statistics reminds its readers, with large data sets, confidence intervals have to be increased, so a 90 per cent confidence level would not then be valid -- the hypothesis is falsified.
>
> I also went to the UNSW book store and looked at all the elementary textbooks on statistics and econometrics. None of them said what Sinclair claimed they did. None of them.
>
> Sinclair, I think you need to send a correction for your article to IPA Review."
>
> Davidson:  "Tim, eye on the ball. I also provided Zellner and Leamer as refs. I have others too, there is a nice example in Walpole, Introduction to Statistics (but he doesn't expand on it), and Rao. But unless you want to argue I fabricated Lindleys paradox (I'm sure I would have called it 'Davidson's paradox'), and the formulas that underlie hypothesis testing (especially the bit about dividing through by the square root of n) I'm not sure where this leaves your argument about the IPCC."
>
> Robson and Davidson then reply with quotes from stats textbooks, claiming that these quotes support their position, as indeed they appear to do. However, Lambert then shows that the quotes have been edited. Try reading the quotes below, skipping the sections in bold, then read them again with the bold text included.
>
> Lambert:  "Elementary stats texts do not say that you have to increase significance levels with large samples and they must definitely do not say that you should reject the alternative hypothesis if the specified level of significance is not achieved. Alex and Sinclair have now resorted to quote doctoring to try to make it appear that they do. In the quotes below I have added in bold the words they left out from their quotes.
>
> Sinclair's quote of Judge et al
>
> "For a given significance level the probability of a Type I error is held constant whatever the sample size, and the probability of a Type II error declines as T increases. This situation favors H0 when T is small and favors H1 when T is large. In contrast, the Bayesian approach treats both errors symmetrically and favors neither hypothesis as T increases. For the sampling theory effect to be achieved within the Bayesian framework, we would need to make the prior odds ration a decreasing function of sample size. Similarly, for the Bayesian effect to be achieved within the sampling theory framework, the significance level would need to be a decreasing function of sample size".
>
> Judge is contrasting Bayesian and frequentist hypothesis testing. He is not, as Sinclair tried to make you think with his quote doctoring, advocating that the significance level needs to decrease with bigger samples. Compare the original with Sinclair's version and ask yourself if he was being honest.
>
> Next we have Alex's quote of Kennedy. Bolded are the bits he left out without even an ellipsis:
>
> "One of the more interesting problems is the fact that almost any parameter can be found to be significantly different from zero if the sample size is sufficiently large. (Almost every relevant independent variable will have some influence, however small, on a dependent variable; increasing the sample size will reduce the variance and eventually make this influence statistically significant.) Thus, although a researcher wants a large sample size to generate more accurate estimates, too large a sample size might cause difficulties in interpreting the usual tests of significance. McCloskey and Ziliak (1996) look carefully at a large number of empirical studies in economics and conclude that researchers seem not to appreciate that statistical significance does not imply economic significance. One must ask if the magnitude of the coefficient in question is large enough for its explanatory variable to have a meaningful (as opposed to "significant") influence on the dependent variable. This is called the too-large sample size problem. It is suggested ...
>
> plus de détails »

Most interesting !
Has any of these arguments stopped the Drought ?

jpturcaud