🚀 go-pugleaf

In article <9011090259.AA00499@wubios.wustl.edu> david@WUBIOS.WUSTL.EDU (David J. Camp) writes:
>Okay, I am ignorant.  What does the "regression" in "regression
>testing" refer to?  -David-

The term "regression" refers to a previously fixed bug recurring in a
later version of the software - the software is said to have
"regressed" to an earlier (and more broken) state of development.  A
strict testing methodology would have you create a test to reproduce
every bug you find in the system.  The resulting suite of tests would
be applied to every new version of the software; this is called
"regression testing".  This isn't usually done, due to the time
required to test stuff manually.  Even the few companies I know of
with good automated test systems (including the one I helped develop)
generally run full regression tests only every few development spins.
Note that regression testing is not just functional testing.  It can
and should cover performance, stress, limit and negative testing as
well.  

Regression test suites tend to be large.  The suite I used for
Unix was around 200M of source and executables.  Guessing a 1:1 ratio
in size of test suite to size of product is a pretty good rule of
thumb, so a full emacs regression suite ought to weigh in around 10M,
for example.

--opinion follows--

To stick my two cents in, I'd say that regression suites for GNU
products are a great idea, but they should be developed by (and
marketed to) the 3rd parties who really need them.  As a "private
citizen", I have no need for a complete functional test for gcc; Data
General may feel differently, since they are distributing it as part
of their products.  They and others like them would probably love to
pay for standardized tests, rather than go to the expense of
developing them in house.

One of the reasons the FSF is so prolific is that it is composed of
hackers (and I don't mean criminals).  In my years as a quality
assurance engineer, I have seen that hacking and rigorous test
development are largely incompatible.  Oh, there are a rare few
individuals whose joy is undiminished by spending long hours thinking
up how to break code, and turning that into tests, but most people who
love to hack would rather be clapped in irons.  I made my living
because *I* like hacking *tests*.

So, my suggestion is that interested people organize themselves, and
start cranking out test suites.  Due to the size issue, I don't think
they should be inseparable parts of the standard distributions.
(Configuration checkers might be a good idea, though.)  Somebody might
even be able to make some money...

--Peter





--
Peter H. Schmidt	| ...harvard!cayman!winter!pschmidt
3 Colonial Village, #10	| winter!pschmidt@cayman.com
Arlington, MA  02174	| -- Speaking for myself.