🚀 go-pugleaf

It depends on how you are failing, and the nature of the nonlinearity.  If the
solution is continuous in slope,  you might explore using full newton-raphson
with line search.  The line search provides a dynamically computed over/under
acceleration factor which can sometimes stablize certain problems, and can
sometimes speed up the rate of convergence.
-- 

Regards.
 
Don Rolph  a722756@pan.mc.ti.com WD3 MS10-13 (508)-699-1263

Typically, Newton-based techniques are local continuation methods and the initial guess
is very important.  In most of the cases, I have used (highly non-linear as well) N-technique
they have converged unless you have cases like turning points, bifurcation points, and so on.
Since you already have an idea of the solution, I suspect that the problem could be due
some small error in your Jacobian.  In most of the cases, double-checking your Jacobian will
be your worthwhile.

S.Shankar
Tech CAD
INTEL Corporation

In article <92Dec17.174419.27750@acs.ucalgary.ca>
mgh@hobbes.phys.ucalgary.ca (Mike Henderson) writes:

>
>I am trying to solve a set of 8 nonlinear equations in 8 unknowns.
>I have tried the Newton-Raphson technique given in NR (I know
>roughly what the solution should be) but havent had much luck. Are
>there any more robust methods to choose from?
>
>Mike Henderson
>Dept. of Physics and Astronomy
>University of Calgary
>mgh@hobbes.phys.ucalgary.ca
>

 I had used the Newton-Raphson method in NR on a set of 5 non-linear
 equations in 5 unknowns.  I was happy to finally get the method to
 converge regulary.

 To get the method to converge on regular basis, it was necessary to
 re-order my equations, by placing:
      -- linear equations first
      -- slightly non-linear equations next
      -- non-linear equations last.

 I found this information in some documentation (???)  that I
 was reading.

 I hope this helps.

 Later,
 Mark

In article <92Dec17.174419.27750@acs.ucalgary.ca> mgh@hobbes.phys.ucalgary.ca (Mike Henderson) writes:
>I am trying to solve a set of 8 nonlinear equations in 8 unknowns.
>I have tried the Newton-Raphson technique given in NR (I know
>roughly what the solution should be) but havent had much luck. Are
>there any more robust methods to choose from?
>
>Mike Henderson
>Dept. of Physics and Astronomy
>University of Calgary
>mgh@hobbes.phys.ucalgary.ca
Is it possible in your case to calculate the analytical derivative?
If so ::::---->
I had similar kinda problem.  I had 10 NLE's and 12 variables 
Initially it was not working because of a few errors in the 
calculations of the analytical derivatives required for the 
Jacobian.  You can check the derivatives by calculating the 
numerical derivatives and they must be very close.If there
is large difference in numerical and analytical derivatives then
you are doing some mistake in calculation of the analytical
derivative or may be some other simple problem such as matrix
size.  Check if your analytical and numerical derivatives are same
I hope it will work because it is working in my case when
the numerical and analytical derivatives were very nearly equal(3 digits


ravi

Perhaps I can be of some help since I developed the code.  It might be
of use to the general community if you would post all your replies.

In article <1992Dec18.193249.22059@murdoch.acc.Virginia.EDU> you write:
>Hi,
>
>  I am thinking about using VFSR (A Simulated Reannealing code from
>  NETLIB) on my problem which is to minimize a nonlinear function
>  subject to nonlinear inequality constraints. In addition, the
>  constraints are nondifferentiable wrt the parameters although
>  the objective function is differentiable.
>
>  I dont know anything about the theory behind VFSR and I wanted to
>  use it as a black box. My question is:
>  Can I use VFSR for this purpose? 

I've used similar constraints.  One case that was published was in
%A L. Ingber
%T Statistical mechanics of neocortical interactions:
A scaling paradigm applied to electroencephalography
%J Phys. Rev. A
%N 6
%V 44
%P 4017-4060
%D 1991

>  It is my understanding that VFSR requires the user to supply a
>  subroutine which returns with the cost fuction and a flag which
>  determines whether the (parameter) state at that point violates some
>  constraints or not. Apparently, if the constraints are violated, a
>  new set of trial parameters are generated. Is there a guarantee that
>  the new set of trial parameters are valid? If not, then it would seem
>  to me that you could get stuck at such points.

Yes, you could get stuck if the space being sampled had no other
good points.  I.e., on return from the cost function with an invalid
flag, a new random point is selected.  If you think your space might be
flooded with invalid points, you might set up some adaptive schedule to
change the ranges during the run.  I have done this in other problems,
e.g., by passing the parameter maximum and minimum values along with
the cost function.  On lines 133-135 in vfsr.h I have placed some
comment for this possibility.

>  I would appreciate any information on the issues above.
>  Thanks.
>
>
>  -sriram srinivasan
>   (ss4i@kelvin.seas.virginia.edu)

Lester
-- 
|| Prof. Lester Ingber                         [10ATT]0-700-L-INGBER ||
|| Lester Ingber Research                        Fax: 0-700-4-INGBER ||
|| P.O. Box 857                           Voice Mail: 1-800-VMAIL-LI ||
|| McLean, VA  22101                EMail: ingber@alumni.caltech.edu ||

In <92Dec17.174419.27750@acs.ucalgary.ca>
	mgh@hobbes.phys.ucalgary.ca (Mike Henderson) writes:

>I am trying to solve a set of 8 nonlinear equations in 8 unknowns.
>I have tried the Newton-Raphson technique given in NR (I know
>roughly what the solution should be) but havent had much luck. Are
>there any more robust methods to choose from?

In <1992Dec17.173036@roper.mc.ti.com>
	a722756@roper.mc.ti.com (W. Donald Rolph) writes:

>It depends on how you are failing, and the nature of the nonlinearity.
>If the solution is continuous in slope,  you might explore using full
>newton-raphson with line search.  The line search provides a dynamically
>computed over/under acceleration factor which can sometimes stablize
>certain problems, and can sometimes speed up the rate of convergence.

I agree with Don, although I am not entirely sure how to proceed in the
general case.  You need a way to detect divergence and, if you see it, a
way to step on the brakes.  A line search works nicely if you can decide
on 1) what you are searching for, and 2) what triggers an alarm.  The
two can be the same.

In my own work, the nonlinear equations arise from the minimization of a
nonquadratic finite-element functional.  This means that the Hessian
matrix is symmetric, and ultimately positive-definite.  Positive-
definite matrices are sooo easy :)

I guard-band the Newton step with a line search for a minimum, with
initial step length 1.0.  I actually look for a zero in the directional
derivative, but I do make sure that the cost function never increases.
The result is absolutely bullet-proof.  It's reasonably fast, too.  The
trickiest problem I've done is the magnetostatics of a transformer
with a hard-saturating core.  The solver has several hiccups, but it
zeroes in on the answer just the same.

A generalization of what works for me would be to search for
(NL residual).(search direction)=0.  Then again, I have it easy.  I know
that my line search need only look forward, and I know that if I go far
enough I will pick up a sign change and bracket a zero.  If your problem
is not a minimization, or if your equations are not even self-adjoint,
various pathological things can happen.  And if your system of equations
has no solution, no amount of armor-plating will give you one!
-- 

--Pierre Asselin, Santa Barbara, California
  pa@verano.sba.ca.us

In article <92Dec17.174419.27750@acs.ucalgary.ca>, mgh@hobbes.phys.ucalgary.ca (Mike Henderson) writes:
|> I am trying to solve a set of 8 nonlinear equations in 8 unknowns.
|> I have tried the Newton-Raphson technique given in NR (I know
|> roughly what the solution should be) but havent had much luck. Are
|> there any more robust methods to choose from?
|> 
|> Mike Henderson
|> Dept. of Physics and Astronomy
|> University of Calgary
|> mgh@hobbes.phys.ucalgary.ca

The problem with the NR method is that it is not  globally convergent (It needs a
good starting point to get to the solution). One method of solving this problem is
to represent this as a minimization problem and use a globally convergent 
non linear programming (unconstrained) algorithm to solve it. I can
give  a reference to this. 

Numerical methods for unconstrained optimization and nonlinear equations by
J.R Dennis and Robert Schnabel, Prentice Hall (1983). page 147. 


Amal de Silva

In <1992Dec22.044947.17093@mel.dit.csiro.au>
	amal@mel.dit.csiro.au (Amal De Silva) writes:

>One method of solving this problem is to represent this as a
>minimization problem[...]

Well, yes, if you can.  Mike didn't say much about his system of
equations;  it may not be derivable from a minimization.
-- 

--Pierre Asselin, Santa Barbara, California
  pa@verano.sba.ca.us