🚀 go-pugleaf

Tried to email but it bounced ...

In article <920124125346.2040177b@rcf.rsmas.miami.edu> you write:
>
>The dangers of using stepwise regression to search for a model with high
>R-squared are well known.  With small samples and noisy data, searching
>for the best-fitting model can overestimate the fit of a predictive
>model, among other problems.
>
>I am concerned about a similar practice, with a different slant, that
>appears occasionally in my field: searching for a model specification
>that gives a "good" distribution of residuals.  The underlying idea is
>that anything other than i.i.d. normal errors suggests violation of the
>assumptions of a linear model.

A suggestion, a question, and a comment on model-formation.

1) The woes of overfitting can be reduced by resampling. In the case of
   regression this means using cross validation to minimising error-of-
   rediction rather than error-of-fit.

   I'm not immediately certain how you would apply cross-validation in
   your case. Unless, say, 10% of your data is enought to give a believable
   measure of goodness-of-fit.

   However, you could use the jack-knife (or the bootstrap) to estimate the
   confidence intervals for your fitted parameters.

   Off the top of my head. You could probably go further, in one of two ways -
   a) Combine the confidence intervals with priors for the parameters in a
      Bayesian analysis.
   b) Move towards a ridge-regression approach. Minimising a criteria of
      fit which combines your goodness-of-fit metric with a size-of-parameters
      metric.

2) I like your idea of looking for good residual distributions (isn't this
   a form of projection pursuit?). I am *very* interested to know which
   goodness-of-fit statistic you use (and for references to the use of this
   approach).

3) The program MARS (from statlib) tries to re-express your data so that it
   fits a linear model better. I haven't tried it yet, and the approach is
   probably best regarded as somewhat experimental. However, if you are still
   uncertain about the form of your model it might provide useful insights
   (or simply confirm your assumptions).

Regards, Peter