🚀 go-pugleaf

I don't remember all the authors and dates, but Pregibon ?  published
several articles on regression diagnostics for logistic regression
(in Biometrika..late 70's, early 80's ?).
As I recall (fuzzily), Pregibon, et al construct the logistic
equivalent of the OLS HAT matrix which is a basic tool for building
diagnostics in the OLS case.


   "Edward A. Frongillo, Jr." <EAFJ@CORNELLA.BITNET> in commenting
on a suggestion by David Gow ....said
>    Following David Gow's point, for the detection of collinearity
> in a logistic regression (or other regression), since the collinearity
> is a characteristics of the X's only, then why can not one simply
> do an OLS regression to use the available programs for assessing
> collinearity?

As an approximation, this suggestion has some merit and is one which
I have actually used with logit, probit, tobit, and their ilk.

If the data suffers from collinearity problems, these will not
be solved by moving from linear to non-linear models...in my
experience they will be exacerbated.  Before running that
logit/probit/tobit equation with umpty-scump RHS variables,
I used to suggest that clients save themselves a lot of
grief by screening via OLS.  Such advice was usually heeded
by the 2nd or 3rd equation in the days before "STEP-WISE
LOGISTIC REGRESSION" appeared on the seen, but after that
the only people who heeded such advice were those with
many many thousands of observations ( n > 100000 ?).


However, it  isn't absolutely correct that the matrix
which is the source of the problem in the OLS case is the same
as that in the logit/probit/tobit case; it's close but not identical to
the OLS case. In OLS one is actually using the X'X matrix  whereas in the
other cases one has something at best like  H = X'W X where W is a
diagonal matrix which *differs from iteration to iteration*. For all of
these models, the equivalent of the X'X matrix is the Hessian which
is essentially an  X'X matrix (moments matrix) but with model dependent
weights applied to each row of X before accumulating the cross-products.

 Randall Olsen gave an exact factorization of the Tobit model
into these parts in a paper in Econometric some 10 or 12
years ago.  For some other model of this sort, the mechanics
of the factorization would be the same.....only the derivation
 of the weights would differ.

I never worked through the algebra/calculus to determine if there is
some simply stated analytic relation between the X'X matrix and
the Hessian (at the solution point), but it seemed to me that it
would be sheer luck if the hessian were to be significantly less
ill-conditioned than the X'X matrix. With the non-linear models,
{multi-collinearity // ill-conditioning} can easily be confounded
with improper convergence since at least some of the indicators
are shared between the two cases (eigvals, condition#).


     -----------------------------------------------
     Mark D. H. Miller    metrmdhm@suvm.acs.syr.edu
     Syracuse University, Research Computing Services
     Syracuse, New York
     -----------------------------------------------