🚀 go-pugleaf

ajayshah@almaak.usc.edu (Ajay Shah) writes:

>Parallel hardware is obviously a way to get revolutionary gains in
>compute power, but Bill Joy's law suggests that single CPUs will give
>tremendous gains without paying any software irritations (they
>certainly have done great in the last five years).

>If this is indeed the case; that most people use workstation-type
>computers, then the argument changes quite a bit.  There may be room
>for a language with simple semantics like fortran, but only for those
>5% of users who want to use parallel hardware.  I suspect most
>scientific computation out there does not know or care about parallel
>hardware.

>        -ans.
>-- 
>Ajay Shah, (213)749-8133, ajayshah@usc.edu
-- 
===============================================================================
Steve (really "D. E.") Stevenson           steve@hubcap.clemson.edu
Department of Computer Science,            (803)656-5880.mabell
Clemson University, Clemson, SC 29634-1906

In article <l9p02vINNorv@almaak.usc.edu> ajayshah@almaak.usc.edu (Ajay Shah)
writes:
>Parallel hardware is obviously a way to get revolutionary gains in
>compute power, but Bill Joy's law suggests that single CPUs will give
>tremendous gains without paying any software irritations (they
>certainly have done great in the last five years).

     I have no idea what Joy's law might be, but if you mean that you
expect the price/performance ratio for single CPUs to keep going down
the way it has been doing the last little while, I believe you are
mistaken.  There is only so much this ratio can improve before
various bits of physics start to interfere.  It's hard to say how close
to those limits we are, but it strikes me that when we see things like
special cooling systems being considered for top-of-the-line
microcomputers, we must be getting close to the point where increased
performance will cost more, even if performance keeps increasing.

>If this is indeed the case; that most people use workstation-type
>computers, then the argument changes quite a bit.  There may be room
>for a language with simple semantics like fortran, but only for those
>5% of users who want to use parallel hardware.  I suspect most
>scientific computation out there does not know or care about parallel
>hardware.

If my previous argument is correct, then your argument is irrelevant;
parallel hardware is simply the next logical step in improving
price/performance ratio.  You may be at least partly right however:
A lot of tasks are simply not very parallelizable so that, even running
on multi-CPU machines, they will hog one processor and leave the others
alone.  There are nevertheless other factors which in my opinion make
Fortran a logical choice for much scientific computing.

	  1) Fortran has an exceedingly shallow learning curve.
	     I'm not saying that you can learn to program well in
	     Fortran faster than you can learn to program well in any
	     other language, but you can write your first useful program
	     faster in Fortran than in most other languages.  Scientists
	     don't write programs as ends in themselves; it is not just
	     that they THINK they have better things to do with their
	     time, they actually DO have better things to do with their
	     time.  This occasionally causes a good deal of trouble, but
	     it is a sociological factor which can't be overlooked and
	     which accounts for a good deal of Fortran's popularity.

          2) Part of the reason that Fortran is so easy to learn is that
	     it provides fairly direct support for the things which a
	     broad class of scientists do routinely.  No other commonly
	     available language handles complex arithmetic right out of
	     the box.  C++ doesn't count because it forces us to deal
	     with C's awful syntactic conventions.  (To quote Carlin:
	     "These are my rules, I make them up!"  More seriously, C
	     and C++ violate criterion (1).  For example, I learned Fortran
	     by writing a library which performs certain quantum chemical
	     calculations.  As far as I know, this library is still in use.
	     I doubt that I could have undertaken such a project in C and
	     completed it in the alloted time without at least some prior
	     experience with the language.)

          3) You can convert Fortran code to C code, but if you want to
	     have any chance of maintaining it, you had better hold on to
	     the Fortran code.  The old libraries and programs are still
	     useful and we can't just throw them away.

     The old "Fortran optimizes better" argument is weak (as Ajay points
out), but it's hardly the only argument around for the continued existence
of the language.

				Marc R. Roussel
                                mroussel@alchemy.chem.utoronto.ca

In article <l9p02vINNorv@almaak.usc.edu> ajayshah@almaak.usc.edu (Ajay Shah) writes:
>In the Great Lwars, one point which commonly comes up is that fortran
>is conceptually lowbrow and hence well-suited for the generation of
>good code, esp. to exploit various parallel architectures.
>
>  [some stuff deleted]
>
>Parallel hardware is obviously a way to get revolutionary gains in
>compute power, but Bill Joy's law suggests that single CPUs will give
>tremendous gains without paying any software irritations (they
>certainly have done great in the last five years).
>
>If this is indeed the case; that most people use workstation-type
>computers, then the argument changes quite a bit.  There may be room
>for a language with simple semantics like fortran, but only for those
>5% of users who want to use parallel hardware.  I suspect most
>scientific computation out there does not know or care about parallel
>hardware.
>

Let me make two comments.  One is overall agreement with your statement
as regards the likely users of massively parallel supercomputers.
The general comment about supercomputing is that 10% of the computing
community uses 90% of the supercomputing power.  I think that is probably
a correct statement, it certainly is within the DoD and DoE laboratories
I have been affiliated with over the years.  The projections for the use
of high performance massively parallel supercomputers preserve that 
user mixture, meaning that only a small minority of people doing
scientific computing will use highly parallel computers.  The outlook
for small parallelism is quite different however.  Consider the
Sparc 10, a workstation by any measure.  It has 4 CPU's in it and is
meant to exploit parallelism on a small scale.  Just as there has not
been a single CPU supercomputer built in years, I stronly suspect that
there will be fewer and fewer single processor workstations in the future.
It is less clear whether the multiprocessors in your workstation will be
something only the operating system will know about or whether they will
be available to the user.  An side comment is that most workstations sit
idle most of the time, which has generated considerable interest in
distributing parallel computing where you use network linked workstations
as the computing nodes.  

My second comment takes issue with your characterization of Fortran as
the language of choice for parallel computing.  Let us assume you meant
Fortran 90, since there is nothing about Fortran 77 that would make it
particularly good for this purpose (the same being true of C and other
procedural languages out there).  Fortran 90 does have array syntax,
which makes data parallelism easier to perform.  However, the trend is
away from data parallelism and toward control parallelism (though many
would argue this.  However, even the guru's of data parallelism, 
Thinking Machines Corporation, are now producing a system which is
at least partly MIMD).  I guess I don't see anything about Fortran 90
which makes control parallelism any easier.  I would argue in fact
that none of the current generation of procedural languages address
parallelism at all well.  Some of the OOPS concepts make it easier,
but only in allowing one to hide the machine architecture from the user.
Certainly they do little to enhance efficiency and the usual reason you
run on a supercomputer is because you are sweating bricks about
speed (otherwise, why not use a workstation).  Functional languages
actually strike me as the ones currently best suited to expressing
parallelism (data or control).  

Having said all that, there is hardly a bit of difference in the
functionality of Fortran 90 and any of the other procedural languages.
Fortran 90 has structures, pointers, recursion, all the things that
people have been trashing on it for not having all these years.  So
I am not sure I see where you are going with your comment, since there
is little about it that I would call "conceptually lowbrow".  That
it is not an OOPS language or a functional language hardly makes it
conceptually lowbrow.  In addition, each of these concepts, OOPS,
procedural, and functional, have their pros and cons.  If you want
to compete in the 90's world of computing, you are going to have to
use a variety of languages, exploting the strength of each when the
problem demands it.

I would make a final comment about Fortran also.  There are now at
least two variants of Fortran 90 being developed which are attempting
to address the issue of parallelism.  One is Fortran D by Ken Kennedy
and company, the other is High Performance Fortran.  I don't think I
would call either of these "lowbrow" languages.

John
-- 
Dr. John K. Prentice                      
Partner, Quetzal Computational Associates    
3200 Carlisle N.E., Albuquerque, NM   87110-1664; 505-889-4543
john@aquarius.unm.edu -or- jkprent@cs.sandia.gov -or- prentice@rufous.cs.unm.edu

In article <1992Aug27.152740.29641@alchemy.chem.utoronto.ca> mroussel@alchemy.chem.utoronto.ca (Marc Roussel) writes:
>In article <l9p02vINNorv@almaak.usc.edu> ajayshah@almaak.usc.edu (Ajay Shah)
>writes:
[....]
>	     with C's awful syntactic conventions.  (To quote Carlin:
>	     "These are my rules, I make them up!"  More seriously, C
>	     and C++ violate criterion (1).  For example, I learned Fortran
>	     by writing a library which performs certain quantum chemical
>	     calculations.  As far as I know, this library is still in use.
>	     I doubt that I could have undertaken such a project in C and
>	     completed it in the alloted time without at least some prior
>	     experience with the language.)
>
>          3) You can convert Fortran code to C code, but if you want to
>	     have any chance of maintaining it, you had better hold on to
>	     the Fortran code.  The old libraries and programs are still
>	     useful and we can't just throw them away.
>

It is a mistake to compare C++ to C.  With a set of good matrix class
libraries, and some experience with array languages, you can begin
programming immediately in C++, not to mention with many times the
productivity of FORTRAN.  C++ is all about interface.  Let a
specialist design the classes (i.e., the interface) and the
scientist/engineer can go from there.  And I don't throw away my
Fortran - I write C++ wrappers for them.  That way I can merrily
factorize and decompose without having to keep track of all those
Fortran subroutine arguments (because the C++ objects take care of all
that), and I don't have to mess with those cryptic function names
(because I can overload function names and the C++ object knows what
to do).  And if I do want to get into object-oriented programming I
can design my own classes, for example, I could design a new data type
that suited my problem.  With a templated matrix class all I would
have to do is provide functions for computing the operations of a new
data type, for example, +, -, *, and /, for interval numbers, and I
would instantly have an array language with interval numbers.  In fact
with not much trouble I could declare a multi-dimensional array of
double precision rational complex interval numbers (if that was what I
wanted).  
   I think it is a huge mistake to try to do numeric programming in C,
but such a judgment doesn't generalize to C++.  

In article <l9p02vINNorv@almaak.usc.edu> ajayshah@almaak.usc.edu (Ajay Shah) writes:
>... I know
>exactly one person who uses parallel hardware.  Every single person I
>know who does scientific computation uses single-CPUs.

This just means you don't know people with big enough problems.

>Parallel hardware is obviously a way to get revolutionary gains in
>compute power, but Bill Joy's law suggests that single CPUs will give
>tremendous gains without paying any software irritations (they
>certainly have done great in the last five years).

There is a limit. Have you noticed that all the screaming "uniprocessors" are
going superscalar? SPARCs are already starting down the multiprocessing road.

Now I'm not trying to tell you that most scientific computing is parallel, or
anything similarly off-planet. But don't think that parallel computing is
going to go away anytime soon. Eventually, you hit the Boltzmann limit and then
the _only_ performance gain is parallelization. Your enthusiasm for single
CPU performance gains just means that you'll hit this wall sooner, rather than
later.
>
> I suspect most
>scientific computation out there does not know or care about parallel
>hardware.

You aren't talking about the major leaguers here, are you? In my experience,
places with graduate programs specializing in scientific computation are also
hotbeds of parallelization research and cross-fertilization.

Later,
Andrew Mullhaupt

] Parallel hardware is obviously a way to get revolutionary gains in
] compute power, but Bill Joy's law suggests that single CPUs will give
] tremendous gains without paying any software irritations (they
] certainly have done great in the last five years).
                                   ^^^^^^^^^^^^^^^
This is not true.  What is true is that low end machines (PC's and 
workstations) have made great strides in the last ten years.  The high
end SINGLE PROCESSORS have not.  For example, a single Cray processor has
gone from about 140 Megaflops (Cray 1) to 1000 Megaflops (Cray C-90) in
the last 12 years (these are peak speeds; as such, there is a lot of
room for argument).  If you define a uniprocessor as having a single
set of functional units, the peak gets even lower, as peak clock speeds
have gone from about 12.5 ns (Cray 1) to 2.9 ns (NEC SX-3).  

Bill Gropp