🚀 go-pugleaf

hougen@uirvlh.csl.uiuc.edu (Darrell Roy Hougen) writes:

>Before people get too confused, just let me add my two cents worth.
>The number of "degrees of freedom" in a problem represents the amount
>of information in the data and is generally equal to the number of
>samples.  Intuitively, as statistics are computed using the data, the
>independent information in the data is gradually "used up" and the
>number of degrees of freedom is reduced.  The information is not
>actually lost, but the statistics that have been computed contain some
>of the information so that the amount of independent information left
>in the data is reduced.

>[example of sample mean deleted]

I have never been very satisfied with this notion of "degrees of
freedom" being equivalent to "pieces of independent information".  The
correspondence works well enough for linear models and independent
observations, as in the example given, but there are alternative
interpretations (in terms of the dimension of orthogonal subspaces)
that do just as well in this setting.

To put out a concrete example that illustrates the nature of my
qualms, consider the following.  Suppose my data consist of a single
number, say X.  Suppose that I am given two pieces of information
about X (but not X itself):

           (1)  I can observe Y := X**2
           (2)  I can observe Z := {  1 if X > 0
                                   { -1    otherwise

How many "pieces of information" do I now have?  Clearly I can recover
X from Y and Z by the formula X = Z * <positive square root of Y>, but
neither Y nor Z alone suffices (excluding the special case where 
X = Y = 0).  Thus Y and Z must each "use up" (at least) one degree of
freedom in the usual accounting.  Thus I must conclude that

       <degrees of freedom in X> .GE. 1 + 1 = 2.

For obvious reasons I do not find this to be an acceptable answer!
What do you think?  What am I missing?  Is there any meaningful
interpretation to "degrees of freedom" in a general theory of
inference?


T. Scott Thompson               thompson@atlas.socsci.umn.edu
Department of Economics
University of Minnesota