Query: Regression on 0's and 1's

Thread View: sci.math.stat

5 messages

[T] Back to Threads [V] Tree View [G] Group Articles

5 total messages Started by Carl M. Kadie Tue, 27 Oct 1992 16:38

Author: Carl M. Kadie
Date: Tue, 27 Oct 1992 16:38

68 lines
2855 bytes

I'm (trying to) design an experiment. The input, x, is a real number.
The output, y, is random, but either 0 or 1. I believe that either x
or x^2 has linear relationship with the expected value of y. How can I
test this and find the best model?

(Can anyone suggest a reference that covers this?)

- Carl


p.s. I've considered three designs, but each has problems or questions.
  
Design 1:
  Run m trials, producing data such as
    { (.34,0), (.53,1), (.13,0), ...}
  Do linear regression on the data.

  Problem:
     Linear regression assumes that y values are distributed
     normally, these y values actually have the Bernoulli distribution,
     which is quite different.

     Lack of fit not tested.

Design 2 -- (Repeated experiments to allow y's to be replaced with sample mean)
   Get m1 values of x
      For each value of x, get m2 measure of y, find the sample mean.
      Producing data such as
      { (.34,.80), (.53,.92), (.13,.76), ...}

   Do linear regression on the data.

   Problem:
       What relationship between m1 and m2 minimizes the number of
       experiments needed?

       Linear regression assumes that y values are distributed
       normally, these y values actually have the Binomial-related
       distribution. When the expect value of y is near 0 or 1,
       a large m2 will be needed to make the Binomial-related
       distribution approximate a normal distribution.
          
       Lack of fit not tested.

Design 3 -- (More repeated experiments to allow lack of fit tests)
   Get m1 values of x
      For each value of x, get m2 measure of y, find the sample mean.
        Do this m3 times.

         Producing data such as
          { (.34,.80), (.34,.82), (.34,.73),  ...
            (.53,.92), (.53,.90), (.53,.98),  ...
            (.13,.76), (.13,.82), (.13,.77),  ...
             ...}   
 Do linear regression (and lack of fit testing) on the data.

   Problem:
       What relationship between m1, m2, and m3 minimizes the number of
       experiments needed?

       Linear regression assumes that y values are distributed
       normally, these y values actually have the Binomial-related
       distribution. When the expect value of y is near 0 or 1,
       a large m2 will be needed to make the Binomial-related
       distribution approximate a normal distribution.

-- 
Carl Kadie -- kadie@cs.uiuc.edu -- University of Illinois at Urbana-Champaign

Re: Query: Regression on 0's and 1's

Author: T. Scott Thompson
Date: Tue, 27 Oct 1992 17:48

29 lines
2036 bytes

kadie@cs.uiuc.edu (Carl M. Kadie) writes:

>I'm (trying to) design an experiment. The input, x, is a real number.
>The output, y, is random, but either 0 or 1. I believe that either x
>or x^2 has linear relationship with the expected value of y. How can I
>test this and find the best model?

Since E(y) is a probability for the event y=1, it must lie in the
interval [0,1].  But all linear and quadratic functions of x
eventually move outside this interval, so your hypothesis cannot be
correct unless the range of x is limited.  Even then, it is not
usually appropriate to fit a linear or quadratic model.  A bounded
(between 0 and 1) monotone transformation of the linear or quadratic
function is usually fit instead.

However, unless you know more about how the data are generated it is
not possible to make a definitive statement about which approaches
"work" and which do not.  You need a more general model of how the
probability for y=1 depends on x in order to construct a formal test
of your specific hypothesis.

Standard approaches to this problem include the binary "logit" and
"probit" models, which differ only in their choice of the monotone
transformation listed above.  There are many, many references.  One
starting point is the book "Analysis of Binary Data" by D.R. Cox.
--
T. Scott Thompson              email:  thompson@atlas.socsci.umn.edu
Department of Economics        phone:  (612) 625-0119
University of Minnesota        fax:    (612) 624-0209

Re: Query: Regression on 0's and 1's

Author: Chuck Haas
Date: Tue, 27 Oct 1992 19:26

25 lines
1530 bytes

In article <BwsGvK.Jpu@cs.uiuc.edu> Carl M. Kadie, kadie@cs.uiuc.edu
writes:
>The output, y, is random, but either 0 or 1. I believe that either x ...


This sounds like a bioassay problem with a quantal response.  There is a
large literature on this problem.  It can be solved using maximum
likelihood estimation (i.e., the binomial response with a mean
probability given by a functional relationship with respect to your 'x'
values.

SYSTAT will solve this sort of problem, providing that you can write an
explicit relationship for the expected response probability in terms of
the "x" values -- in fact, EXCEL ver 3.0 or higher (with the solver add
in) can also be set up to solve such a problem.





Charles N. Haas
Professor, Environmental Engineering
Drexel University, Philadelphia PA 19104
======================================
CHaas@COE.Drexel.Edu

Re: Query: Regression on 0's and 1's

#11

Author: Jerry Dallal
Date: Tue, 27 Oct 1992 19:39

11 lines
932 bytes

In article <BwsGvK.Jpu@cs.uiuc.edu>, kadie@cs.uiuc.edu (Carl M. Kadie) writes:
> I'm (trying to) design an experiment. The input, x, is a real number.
> The output, y, is random, but either 0 or 1. I believe that either x
> or x^2 has linear relationship with the expected value of y. How can I
> test this and find the best model?
> 
> (Can anyone suggest a reference that covers this?)
> 
> - Carl

R.A. Fisher called it "discriminant analysis."

Re: Query: Regression on 0's and 1's

#12

Author: Gordon H. Fick
Date: Tue, 27 Oct 1992 23:06

28 lines
1791 bytes

>>The output, y, is random, but either 0 or 1. I believe that either x ...
>
>
>This sounds like a bioassay problem with a quantal response.  There is a
>large literature on this problem.  It can be solved using maximum
>likelihood estimation (i.e., the binomial response with a mean
>probability given by a functional relationship with respect to your 'x'
>values.
>
>SYSTAT will solve this sort of problem, providing that you can write an
>explicit relationship for the expected response probability in terms of
>the "x" values -- in fact, EXCEL ver 3.0 or higher (with the solver add
>in) can also be set up to solve such a problem.
>
>
Splus is ideal for binary/binomial regression with logit or
probit links. Superb for diagnostic displays and other graphics.

GLIM is still a fine alternative especially if there are a number
of candidate models in the X variables

EGRET is a fine menu driven alternative... and it will do
conditional logistic regression along with the standard stuff.


-- 
Gordon H. Fick - Department of Community Health Sciences
University of Calgary - Calgary, Alberta CANADA T2N 4N1 - (403) 220-6939

Thread Navigation

This is a paginated view of messages in the thread with full content displayed inline.

Messages are displayed in chronological order, with the original post highlighted in green.

Use pagination controls to navigate through all messages in large threads.

Back to All Threads