🚀 go-pugleaf

Dear Phil -- You haven't told us enough to enable us to validate or
invalidate your advice.  There are lots of things that can go wrong with
logistic regression, especially with 6-8 predictors and only 34
observations.  An a priori likely explanation of the problem is that the
predictor variables fall into two distinct 'clouds' in the space of
predictor variables, and that one cloud is nearly all 'successes' and the
other is nearly all 'failures.'  If this were the case, then for a certain
linear combination of predictors z = b1*x1 + ... +b8*x8  a graph of response
vs. z would look like this:

      1 |                             *  *   *** **
        |
        |
        |
      0 |     *  * *   *   * *   **
        ------------------------------------------------> z

If this were the case, the slope in the logistic regression of Y vs. z would
go to infinity, and so would the intercept, while their ratio converged to a
constant.  In terms of the original 8 predictors, some regression
coefficients would go to +/- infinity, some might stay finite or just wander
aimlessly, and the intercept would do who knows what.
        Another possibility is a severe multicollinearity in the
predictors.  This would manifest itself as aimless wandering of all iterates
of the regression coefficients, since big changes in the b's would have
negligible effect on the logistic likelihood.
        In view of all this, it doesn't seem adequate to tell the student to
'take the money and run.'  It isn't clear whether the money is real or
counterfeit.  What can be done is this:  look for multicollinearity among
the predictors, and if it is present in gross amounts, reduce the predictor
set accordingly.  Note that with lots of variables, many moderate
correlations can indicate some complex linear dependence among the x's.  A
principal component analysis among the x's might be in order.  A cluster
analysis might help to pick up the 'separate clouds of points' problem if it
is there.  Creating a new variable to represent the clustering might be
helpful.  Of course, any new variables that you create should have
scientific meaning.
        Good luck to you and your student.

Paul J. SMITH, Mathematics Department, University of Maryland at College Park
Email:Paul_J_SMITH@umail.umd.edu (ps26)
Phone: (301) 405-5104