🚀 go-pugleaf

diamong@gov.on.ca (Gerald Diamond) writes:

>I am looking for a good reference (or references) for 
>a least-squares type algorithm which treats both X and Y as 
>having errors rather than the usual case of assuming all the
>errors are in one variable.

	Try the book "Data Reduction and Error Analysis for the Physical
Sciences"  by Philip R Bevington.  It was printed in 1969, and just this
year, one of his colleagues updated the book.  The 1992 version comes with
a disk of the source code in Pascal.  Both versions have the Fortran code.
I just used the book to write a nonlinear least sqaures program using the
Levenberg-Marquardt algorithm.  Works great!

Magdi N. Azer
magdi@uxh.cso.uiuc.edu

magdi@uxh.cso.uiuc.edu (Magdi N Azer) writes:

>diamong@gov.on.ca (Gerald Diamond) writes:

>>I am looking for a good reference (or references) for 
>>a least-squares type algorithm which treats both X and Y as 
>>having errors rather than the usual case of assuming all the
>>errors are in one variable.

>	Try the book "Data Reduction and Error Analysis for the Physical
>Sciences"  by Philip R Bevington.  It was printed in 1969, and just this
>year, one of his colleagues updated the book.  The 1992 version comes with
>a disk of the source code in Pascal.  Both versions have the Fortran code.
>I just used the book to write a nonlinear least sqaures program using the
>Levenberg-Marquardt algorithm.  Works great!

The 1969 Bevington copy only says that the usual ordinary linear squares can
be used if the errors in x are sufficiently small to ignore and does not offer
anything for the case where the x and y errors are comparable. Some
straight-forward articles for the case where both variables are subject to
errors are the following:

Mandel, John (1984) "Fitting Straight Lines When Both Variables are Subject to
Error", Journal of Quality Technology, v. 16(1):1.

Reilman, Miriam, Richard Gunst, and Mani Lakshminarayanan (1986) "Stochastic
Regression with Errors in Both Variables", Journal of Quality Technology,
v. 18(3): 162.

Feigelson, Eric and Gutti Babu (1992) "Linear Regression in Astronomy II",
Astrophysical Journal--will appear in Sept 20, 1992 (I have a preprint).

This subject has been the topic of heated debate for decades. The last
reference has a listing of the important papers in various disciplines in
its bibliography section. Code can be received by sending a request to
code@stat.psu.edu. Be aware that what algorithm you use depends on the
error structure of the (x,y) values and whether or not you want to find the
slope of the line or you want to predict a y given a x.

nick
strobel@dirac.phys.washington.edu

In article <1992Aug4.135256.11081@gov.on.ca> diamong@gov.on.ca (Gerald Diamond) writes:
>I am looking for a good reference (or references) for 
>a least-squares type algorithm which treats both X and Y as 
>having errors rather than the usual case of assuming all the
>errors are in one variable.

If you are looking for something which has a meaning from the 
standpoint of statistical inference, there are things which can
be done, but not simply.  
-- 
Herman Rubin, Dept. of Statistics, Purdue Univ., West Lafayette IN47907-1399
Phone: (317)494-6054
hrubin@pop.stat.purdue.edu (Internet, bitnet)  
{purdue,pur-ee}!pop.stat!hrubin(UUCP)

In article <1992Aug4.135256.11081@gov.on.ca> diamong@gov.on.ca (Gerald Diamond) writes:
>I am looking for a good reference (or references) for 
>a least-squares type algorithm which treats both X and Y as 
>having errors rather than the usual case of assuming all the
>errors are in one variable.

Since this problem occurs frequently, I am taking the liberty of
appending a rather long list of references from my personal data base.
It is the result of a brute-force search on keywords and certainly
contains some irrelevant articles.  With equal certainty it also
omits some relevant articles.

Pete Stewart

\begin{thebibliography}{10}

\bibitem{abde:74}
N.~N. Abdelmalek.
\newblock On the solution of least squares problems and pseudo-inverses.
\newblock {\em Computing}, 13:215--228, 1974.

\bibitem{ande:76}
T.~W. Anderson.
\newblock Estimation of linear functional relationships: Approximate
  distributions and connections with simultaneous equations in econometrics.
\newblock {\em Journal of the Royal Statistical Society}, 38:1--31, 1976.

\bibitem{ande:84}
T.~W. Anderson.
\newblock Estimating linear statistical relationships.
\newblock {\em Annals of Statistics}, 12:1--45, 1984.

\bibitem{ardr:89}
M.~Arioli, I.~S. Duff, and P.~P.~M. van Rijk.
\newblock On the augmented system approach to sparse least-squares problems.
\newblock {\em Numerische Mathematik}, 55:667--685, 1989.

\bibitem{berb:76}
A.~E. Beaton, D.~B. Rubin, and J.~L. Barone.
\newblock The acceptability of regression solutions: Another look at
  computational accuracy.
\newblock {\em Journal of the American Statistical Association}, 71:158--168,
  1976.

\bibitem{bels:82}
D.~A. Belsley.
\newblock Assessing the presence of harmful collinearity and other forms of
  weak data through a test for signal-to-noise.
\newblock {\em Journal of Econometrics}, 20:211--253, 1982.

\bibitem{bels:84}
D.~A. Belsley.
\newblock Demeaning conditioning diagnostics through centering.
\newblock {\em The American Statistician}, 38:73--77, 1984.

\bibitem{bekw:80}
D.~A. Belsley, A.~E. Kuh, and R.~E. Welsch.
\newblock {\em Regression Diagnostics: Identifying Influential Data and Sources
  of Collinearity}.
\newblock John Wiley and Sons, New York, 1980.

\bibitem{berk:50}
J.~Berkson.
\newblock Are there two regressions.
\newblock {\em Journal of the American Statistical Association}, 45:164--180,
  1950.

\bibitem{bjor:88a}
{\AA}.~Bj{\"o}rck.
\newblock Error analysis of least squares algorithms.
\newblock Technical Report LiTH-MAT-R-1988-22, Department of Mathematics,
  Link{\"o}ping University, 1988.

\bibitem{bjor:89}
{\AA}.~Bj{\"o}rck.
\newblock Componentwise backward errors and condition estimates for linear
  least square problems.
\newblock Technical Report LiTH-MATH-R-1989-13, Department of Mathematics,
  Link{\"o}ping University, 1989.

\bibitem{bjor:91a}
{\AA}.~Bj{\"o}rck.
\newblock Component-wise perturbation analysis and errors bounds for linear
  least square solutions.
\newblock {\em BIT}, 31:238--244, 1991.
\newblock Cited in {\AA ke Bj\"orck's} bibliography on least squares, which is
  available by anonymous ftp from {\tt math.liu.se} in {\tt pub/references}.

\bibitem{brkr:74}
G.~F. Brown, J.~B. Kadane, and J.~G. Ramage.
\newblock The asymptotic bias and mean-squared error of double {$K$}-class
  estimators when the distrubances are small.
\newblock {\em International Economic Review}, 15:667--679, 1974.

\bibitem{dast:88}
Nancy David and G.~W. Stewart.
\newblock Hypothesis testing with errors in the variables.
\newblock Technical Report TR-1735, University of Maryland Department of
  Computer Science, 1988.

\bibitem{dahu:75}
R.~B. Davies and B.~Hutton.
\newblock The effects of errors in the independent variables in linear
  regression.
\newblock {\em Biometrika}, 62:383--391, 1975.

\bibitem{demm:88}
J.~Demmel.
\newblock The probability that a numerical analysis problem is difficult.
\newblock {\em Mathematics of Computation}, 50:449--480, 1988.

\bibitem{elde:80}
L.~Eld\'en.
\newblock Perturbation theory for the least squares problem with linear
  equality constraints.
\newblock {\em SIAM Journal on Numerical Analysis}, 17:338--350, 1980.

\bibitem{feld:74}
M.~Feldstein.
\newblock Errors in variables: A consistent estimator with smaller {MSE} in
  finite samples.
\newblock {\em Journal of the American Statistical Association}, 69:990--996,
  1974.

\bibitem{full:87}
W.~A. Fuller.
\newblock {\em Measurement Error Models}.
\newblock John Wiley, New York, 1987.

\bibitem{gall:81}
P.~P. Gallo.
\newblock Consistency of regression estimates when some variables are subject
  to error.
\newblock Mimeo Series 1328, Institute of Statistics, University of North
  Carolina at Chapel Hill, 1981.

\bibitem{gera:76}
V.~J. Geraci.
\newblock Identification of simultaneous equation models with measurement
  error.
\newblock {\em Journal of Econometrics}, 4:263--283, 1976.

\bibitem{gera:77}
V.~J. Geraci.
\newblock Estimation of simultaneous equation models with measurement error.
\newblock {\em Econometrica}, 45:1243--1255, 1977.

\bibitem{gope:73}
G.~H. Golub and V.~Pereyra.
\newblock The differentiation of pseudoinverses and nonlinear least squares
  problems whose variables separate.
\newblock {\em SIAM Journal on Numerical Analysis}, 10:413--432, 1973.

\bibitem{gope:76}
G.~H. Golub and V.~Pereyra.
\newblock Differentiation of pseudoinverses, separable nonlinear least squares
  problems and other tales.
\newblock In M.~Z. Nashed, editor, {\em Generalized Inverses and Applications},
  pages 303--324, New York, 1976. Academic Press.

\bibitem{govl:80}
G.~H. Golub and C.~F. Van~Loan.
\newblock An analysis of the total least squares problem.
\newblock {\em SIAM Journal on Numerical Analysis}, 17:883--893, 1980.

\bibitem{gowi:66}
G.~H. Golub and J.~H. Wilkinson.
\newblock Note on the iterative refinement of least squares solution.
\newblock {\em Numerische Mathematik}, 9:139--148, 1966.

\bibitem{gril:74}
Z.~Griliches.
\newblock Errors in variables and other unobservables.
\newblock {\em Econometrica}, 42:971--998, 1974.

\bibitem{high:90}
N.~J. Higham.
\newblock Computing error bounds for regression problems.
\newblock In P.~J. Brown and W.~A. Fuller, editors, {\em Contemporary
  Mathematics 112: Statistical Analysis of Measurement Error Models and
  Applications}, pages 195--210, Providence, Rhode Island, 1990. American
  Mathematical Society.

\bibitem{homo:72}
S.~D. Hodges and P.~G. Moore.
\newblock Data uncertainties and least squares regression.
\newblock {\em Applied Statistics}, 21:185--195, 1972.

\bibitem{kada:71}
J.~B. Kadane.
\newblock Comparison of $k$-class estimators when the disturbances are small.
\newblock {\em Econometrica}, 39:723--737, 1971.

\bibitem{kasw:92}
R.~Karlsson, J.-G. Sun, and B.~Wald\`en.
\newblock Optimal backward perturbation bounds for the linear least squares
  problem.
\newblock Technical Report LiTH-MAT-R-1992-06, Department of Mathematics,
  Link{\"o}ping University, 1992.
\newblock Cited in {\AA ke Bj\"orck's} bibliography on least squares, which is
  available by anonymous ftp from {\tt math.liu.se} in {\tt pub/references}.

\bibitem{koni:81}
H.~S. Konijn.
\newblock Maximum likelihood estimators and confidence intervals for a simple
  errors in variables model.
\newblock {\em Communications in Statistics: Part A}, 10:983--996, 1981.

\bibitem{kumm:1879}
C.~H. Kummell.
\newblock Reduction of observation equations which contain more than one
  observed quantity.
\newblock {\em The Analyst}, 6:97--105, 1879.

\bibitem{laha:74}
C.~L. Lawson and R.~J. Hanson.
\newblock {\em Solving Least Squares Problems}.
\newblock Prentice Hall, Englewood Cliffs, New Jersey, 1974.

\bibitem{lind:47}
D.~V. Lindley.
\newblock Regression lines and the linear functional relationship.
\newblock {\em Supplement to the Journal of the Royal Statistical Society},
  9:218--244, 1947.

\bibitem{lots:83}
P.~L{\"o}tstedt.
\newblock Perturbation bounds for the linear least squares problem subject to
  linear inequality constraints.
\newblock {\em BIT}, 23:500--519, 1983.

\bibitem{mada:59}
A.~Madansky.
\newblock The fitting of straight lines when both variables are subject to
  error.
\newblock {\em Journal of the American Statistical Association}, 54:173--205,
  1959.
\newblock Cited in \cite{govl:89}.

\bibitem{marq:80}
D.~W. Marquardt.
\newblock Comment on {Smith} and {Campbell} (1980).
\newblock {\em Journal of the American Statistical Association}, 75:87--91,
  1980.

\bibitem{mora:71}
P.~A.~P. Moran.
\newblock Estimating structural and functional relationships.
\newblock {\em Journal of Multivariate Analysis}, 1:232--255, 1971.

\bibitem{paar:79}
M.~Paardekooper.
\newblock Inverse perturbations for approximations of least squares solutions.
\newblock Technical Report 79.083, Katholieke Hogeschool Tilburg, Faculteit der
  Economische Wetenschappen, 1979.

\bibitem{paig:79}
C.~C. Paige.
\newblock Computer solution and perturbation analysis of generalized linear
  least squares problems.
\newblock {\em Mathematics of Computation}, 33:171--184, 1979.

\bibitem{reie:50}
O.~Reiers{\o}l.
\newblock Identifiability of a linear relation between variables which are
  subject to error.
\newblock {\em Econometrica}, 18:375--389, 1950.

\bibitem{schm:79}
K.~C. Schmidt.
\newblock Selecting subsets of variables in the linear regression problem.
\newblock Master's thesis, Department of Mathematics, University of Maryland,
  College Park, 1979.

\bibitem{scti:85}
H.~Schwetlick and V.~Tiller.
\newblock Numerical methods for estimating parameters in nonlinear models with
  errors in the variables.
\newblock {\em Technometrics}, 27:17--24, 1985.
\newblock Cited in {\AA ke Bj\"orck's} bibliography on least squares, which is
  available by anonymous ftp from {\tt math.liu.se} in {\tt pub/references}.

\bibitem{spre:66}
P.~Sprent.
\newblock A generalized least-squares approach to linear functional
  relationships.
\newblock {\em Journal of The Royal Statistical Society, Series B},
  28:278--297, 1966.

\bibitem{sriv:72}
V.~K. Srivastava.
\newblock Disturbance-variance estimation in simultaneous equations when
  disturbances are small.
\newblock {\em Journal of the American Statistical Association}, 67:164--168,
  1972.

\bibitem{stew:69}
G.~W. Stewart.
\newblock On the continuity of the generalized inverse.
\newblock {\em SIAM Journal on Applied Mathematics}, 17:33--45, 1969.

\bibitem{stew:77a}
G.~W. Stewart.
\newblock On the perturbation of pseudo-inverses, projections, and linear least
  squares problems.
\newblock {\em SIAM Review}, 19:634--662, 1977.

\bibitem{stew:77b}
G.~W. Stewart.
\newblock Sensitivity coefficients for the effects of errors in the independent
  variables in a linear regression.
\newblock Technical Report TR-571, Computer Science Department, University of
  Maryland, 1977.

\bibitem{stew:83}
G.~W. Stewart.
\newblock A generalization of the {Eckart-Young} approximation theorem.
\newblock Technical Report TR-1325, Department of Computer Science, the
  University of Maryland, 1983.

\bibitem{stew:83a}
G.~W. Stewart.
\newblock A nonlinear version of {Gauss's} minimum variance theorem with
  applications to an errors-in-the-variables model.
\newblock Technical Report TR-1263, Department of Computer Science, University
  of Maryland, 1983.

\bibitem{stew:87d}
G.~W. Stewart.
\newblock Collinearity and least squares regression.
\newblock {\em Statistical Science}, 2:68--100, 1987.

\bibitem{stew:90}
G.~W. Stewart.
\newblock Perturbation theory and least squares with errors in the variables.
\newblock In P.~J. Brown and W.~A. Fuller, editors, {\em Contemporary
  Mathematics 112: Statistical Analysis of Measurement Error Models and
  Applications}, pages 171--181, Providence, Rhode Island, 1990. American
  Mathematical Society.

\bibitem{stew:90d}
G.~W. Stewart.
\newblock Stochastic perturbation theory.
\newblock {\em SIAM Review}, 32:576--610, 1990.

\bibitem{swbo:72}
B.~F. Swindel and D.~R. Bower.
\newblock Rounding errors in the independent variables in a general linear
  model.
\newblock {\em Technometrics}, 14:215--218, 1972.

\bibitem{vdsl:69}
A.~van~der Sluis.
\newblock Condition numbers and equilibration of matrices.
\newblock {\em Numerische Mathematik}, 14:14--23, 1969.

\bibitem{vdsl:75}
A.~van~der Sluis.
\newblock Stability of the solutions of linear least squares problems.
\newblock {\em Numerische Mathematik}, 23:241--254, 1975.

\bibitem{vhva:87a}
S.~Van~Huffel and J.~Vandewalle.
\newblock Subset selection using the total least squares approach in
  collinearity problems with errors in the variables.
\newblock {\em Linear Algebra and its Applications}, 88/89:695--714, 1987.
\newblock Cited in \cite{govl:89}.

\bibitem{waks:92}
B.~Wald\'en, R.~Karlson, and J.-G. Sun.
\newblock Optimal backward perturbation bounds for the linear least squares
  problem.
\newblock Technical Report LiTH-MAT-R-1992-06, Department of Mathematics,
  Link{\"o}ping University, 1992.

\bibitem{wedi:73a}
P.-{\AA}. Wedin.
\newblock On the almost rank deficient case of the least squares problem.
\newblock {\em BIT}, 13:344--354, 1973.

\bibitem{wedi:78}
P.-{\AA}. Wedin.
\newblock Perturbations of the pseudo-inverse and the linear least squares
  problem analysed with suitable matrix decompositions.
\newblock Technical Report UMINF--69.78, Institute of Information Processing,
  University of Ume\aa, 1978.

\bibitem{wedi:85}
P.-{\AA}. Wedin.
\newblock Perturbation theory and condition numbers for generalized and
  constrained linear least squares problems.
\newblock Technical Report UMINF--125.85, Institute of Information Processing,
  University of Ume\aa, 1985.

\bibitem{wedi:87}
P.-{\AA}. Wedin.
\newblock Perturbation theory and condition numbers for generalized and
  constrained least squares problems.
\newblock Technical Report S-901-87, Institute of Information Processing,
  University of Ume\aa, 1987.

\bibitem{wei:90}
M.~Wei.
\newblock Perturbation of the least squares problem.
\newblock {\em Linear Algebra and Its Applications}, 141:177--182, 1990.

\end{thebibliography}

>In article <1992Aug4.135256.11081@gov.on.ca> diamong@gov.on.ca (Gerald Diamond) writes:
>I am looking for a good reference (or references) for 
>a least-squares type algorithm which treats both X and Y as 
>having errors rather than the usual case of assuming all the
>errors are in one variable.
>

  The problem (with arbitrary number of coordinates with errors) is
  discussed and solved in

   A. Celmins, Least-Squares Optimization with Implicit Model Functions, in
   "Mathematical Programming with Data Perturbations II", Anthony V. Fiacco,
   Ed., Marcel Dekker, 1983, pp. 131-152.

   Aivars Celmins