🚀 go-pugleaf

ajayshah@almaak.usc.edu (Ajay Shah) writes:
: 
: I think this strategy is going to revolutionise the way we actually do
: nonlinear maximisations.  I think it's going to obsolete algorithms
: like the BFGS and conjugate gradients etc. because with this in hand
: we get exact hessians and higher.

In most of my problems, the number of variables I'm minimizing over is
quite large, similarly for neural networks.  Don't "BFGS" and Hessian
methods require O(N^2) storage space, at least, in contrast to 
O(N) conjugate gradient.  I don't know what the speed is, but when
N=200 say, the internal matrix multipies that are normally considered trivial
next to function evaluations might not be so trivial.

:  We get large gains in efficiency by
: using higher derivatives (see the Kalaba-Tisher paper for example).

If you have something other than a simple quadratic minimum does this
really do that much good?

: We will endup replacing the status quo of libraries like NPSOL by
: something totally different, and the bonus is that the replacement is
: not in fortrash!  I like it!

Will C++ be able to do:
	A = B*C + D

with A,C,D vectors, and B a matrix  without any copying and creation
of intermediate variables?  (i.e. see Matrix * Vector--- create a new
result vector.  Do + operation on 2 vectors, make another new vector.
Now do the "=" operation and copy it into A)

Perhaps the problem is that C++ let's you define overloaded operators to
do anything, without regard to the normal "rules of algebra", so that it
must assume less.
That's one of the reason's I'm not using it now.

: 
:         -ans.
: -- 
: Ajay Shah, (213)749-8133, ajayshah@usc.edu

--
-Matt Kennel  		mbk@inls1.ucsd.edu
-Institute for Nonlinear Science, University of California, San Diego