🚀 go-pugleaf

In article <1992Sep3.064230.42744@kuhub.cc.ukans.edu> jeff@kuhub.cc.ukans.edu (Jeff Bangert) writes:
>A chemist has asked me to solve a problem: she has data and a model
>which is to be fit by 'least squares'.  It looks like non-linear
>regression, except that:

>    1. the model has two equations
>    2. both are non-linear
>    3. there are parameters common to the two equations.

>I would like to know:

>    1. is there a 'standard' method for solving this problem?
>    2. is there literature in stat or chemistry which I could read?

To use a reasonable statisical approach, you should set up your probablity
model, and carefully assess your assumptions.  Then set up at least a 
reasonable approximation to the likelihood function, and do whatever
processing your assumptions call for.

I would not expect any package to handle this well.  Numerical analysis
is an art.
-- 
Herman Rubin, Dept. of Statistics, Purdue Univ., West Lafayette IN47907-1399
Phone: (317)494-6054
hrubin@pop.stat.purdue.edu (Internet, bitnet)  
{purdue,pur-ee}!pop.stat!hrubin(UUCP)

In article 42744@kuhub.cc.ukans.edu, jeff@kuhub.cc.ukans.edu (Jeff Bangert) writes:
>A chemist has asked me to solve a problem: she has data and a model
>which is to be fit by 'least squares'.  It looks like non-linear
>regression, except that:
>
>    1. the model has two equations
>    2. both are non-linear
>    3. there are parameters common to the two equations.
>
>I would like to know:
>
>    1. is there a 'standard' method for solving this problem?
>    2. is there literature in stat or chemistry which I could read?
>
>Clearly, I can form the sum of squared errors for each equation and
>add them together.  I have lots of methods for minimizing the result.
>However, this seems to ignore the question of the weighting of the two
>equations.
>
>I solved this kind of problem several years ago by writing FORTRAN +
>IMSL.  Now that it has come up again, I wish I had some backup
>literature that at least suggests the 'standard' solution method.  I
>don't like reinventing the wheel.
>
>Since the current data matrix is small, 4 by 2, I'm going to use
>non-linear optimization in Quattro Pro this time.  It produces nice
>graphs.  But, if I stick with QP, I'll also have to calculate the
>standard errors of the parameter estimates.  Right now, I'm not
>looking forward to this.
>
>Thanks for the help,
>-- 
>Jeff Bangert			jeff@kuhub.cc.ukans.edu
>Computer Center			jeff@ukanvax.bitnet
>University of Kansas
>Lawrence, KS 66045
>(913)864-0466


It's a little hard to know the exact formulation of the problem from your post, but if you are referring to 2 equations which arise from chemical kinetics of coupled reactions, you won't get the proper answer by adding the sum of squared errors from the two equations.  This is because the errors for the two equations are most likely different, which would require some weights to be applied, - but the proper weighting is unknown.  Further, the errors for the two responses  are likely to be correlated to an 

unknown extent.  Box and Draper (1965) showed that what should be minimized is the determinant of z'z, where z is the residual matrix for the set of experiments.   They derived this result by Bayesian arguments using a non-informative prior on the coefficents distribution.  Kang (1988) showed that this determinant criterion is also the maximum likelihood solution.  

There also is other literature on this multiresponse problem, so you should be able to find other articles.  Try Technometrics. 

Box and Draper (1965)  "The Bayesian Estimation of Common Parameters from Several Responses" Biometrika, 52, 355.

Kang (1988) "Topics in Multiresponse Regression Analysis", Ph.D. Thesis, U. Wisconsin, Madison.

Martin

In article <1992Sep3.064230.42744@kuhub.cc.ukans.edu> jeff@kuhub.cc.ukans.edu (Jeff Bangert) writes:
>A chemist has asked me to solve a problem: she has data and a model
>which is to be fit by 'least squares'.  It looks like non-linear
>regression, except that:
>
>    1. the model has two equations
>    2. both are non-linear
>    3. there are parameters common to the two equations.
>
>I would like to know:
>
>    1. is there a 'standard' method for solving this problem?
>    2. is there literature in stat or chemistry which I could read?
>

  The problem is to minimize the properly weighted sum of squares of the
  corrections ( residuals) of the *observations* (not the defects of the
  equations!) subject to the constraints given by the model equation
  system.   This leads to a constrained leaast-squares minimization.

  The minimization (least squares with model equations in form of simultaneous
  nonlinear equations) is discussed and an algorithm for the solution
  proposed in
 

   A. Celmins, Least-Squares Optimization with Implicit Model Functions, in
   "Mathematical Programming with Data Perturbations II", Anthony V. Fiacco,
   Ed., Marcel Dekker, 1983, pp. 131-152.

 
  Aivars Celmins
  celms@brl.mil
  (410)-278-6986