🚀 go-pugleaf

In <9011040255.AA23968@wubios.wustl.edu> david@WUBIOS.WUSTL.EDU (David J. Camp) writes:

>I suspect that the largest obstacle to a full GNU operating system
>will be the development of device drivers for the many systems
>to be supported.  The individual manufacturers have a great 
>advantage to being "in-house" to get the necessary information
>to write these drivers.  I suspect that any contributions in this
>are would be welcome.  

>I think a priority should be for a "device driver skeleton" to
>be developed, to streamline the process of developing additional
>drivers.

>david@wubios.wustl.edu             ^     Mr. David J. Camp
>david%wubios@wugate.wustl.edu    < * >   +1 314 382 0584
>...!uunet!wugate!wubios!david      v     "God loves material things."

The GNU development effort is different from typical commercial vendors in
that it's largely "user contributed" and comes "with no warranty, expressed
or implied." In short, no company rests its reputation on the quality of GNU
software.

The result is that the software is strong on features, but sort of beta-test
quality. The maintainers typically respond outstandingly to bug reports and
feature requests, but folks who download GNU code still have to count on a 
certain amount of mucking about before everything installs and runs correctly.

Enough complaining. What I propose is that the GNU project put more emphasis
on automated regression testing. Each piece of software that is release should
have a body of tests released with it. In practice, these tests are more useful
than documentation in providing specifications for a product.

In a world where specifications are given as automated tests, developers on
a wide variety of platforms have a measure of the quality and stability of
their code. If I ported gcc to a new machine, and gcc had some 1000 standard
test cases, I could release my code and say, "I'm releasing this to see if it
would be useful to many people, but it still fails 170 of the GNU gcc v1.0
tests."

Also, installation could be ameliorated by a body of tests. Imagine:
1. I ftp foo.tar.Z, uncompress, and tar into myacct/foo
2. I read the README
3. > make download-test
   all files for GNU foo v1.0 present in myacct/foo
4. > make ARCH=Sun4 build-environment-test
	... makefile checks for necessary include files, utilities, etc. ...
   Sun4 build environment appears to be present
5. > make local-test
   gcc -o blah blah blah
	... makefile builds executable in current directory ...
   gnutest BINDIR=. foo_v1.0.tests
	... gnu test tool runs foo through its paces ...
   gnutest: test failed in file: foo_v1.0.tests line: 25
	expected results:	1.0 1.0 1.0
	actual results:		1.0 NaN -0.999567
   gnutest: floating point exception. core dumped
   gnutest: in file: foo_v1.0.tests line: 173
6. I muck with the source a bit
7. > make local-test
   gcc -o blah blah
   gnutest BINDIR=. foo_v1.0.tests
   gnutest: foo_v1.0.tests complete
8. > make install-test
   ln -s foo /usr/local/bin/foo
   gnutest BINDIR=/usr/local/bin foo_v1.0.tests
   gnutest: foo_v1.0.tests complete

My suspicion is that the FSF uses tools like this in house, but the amount
of effort to get them ready for release has been prohibitive. Until you can
trust the test driver, it's worthless. Where's the bug? In the new software
or the test driver? Or maybe in Sun's new OS? And this kind of thing is 
particularly difficult to make portable.

I just think that an established testing methodology and tools to support it
is essential to the GNU project. These tests can be the last word in
specifications for the software. They can improve the maintenance process, and
they can increase the confidence users have in the software they install.

They add a significant amount of work to the development effort, but I really
think it pays off.

Comments?