🚀 go-pugleaf

In article <26685@winchester.mips.COM> mash@mips.COM (John Mashey) writes:

>6) One possibility, that actually might of marginal use, is to
>have an OS option that (in UNIX, for example), RANDOMIZED .bss and
>sbrked storage.  If you then ran your program several times,
>and it did the same thing, your confidence level would be raised.
>(Something like that, for example, is used by chip designers to find
>bugs.  For example, one of our guys did a program that generated
>"random" self-checking floating-point programs.  It found quite a
>few bugs... Other companies do something similar, I believe.)

    Yes, it is often useful to use randomly generated data for testing
computationally intensive ICs.  (Chips that merely push data around,
rather than operating on it, such as communications chips and
formaters, aren't helped as much because the data path is generally
much simpler.)  I've recently worked on several algorithmic chips (the
Stac 9707 Reed-Solomon ECC chip, and 9703/9704 Data compression
chips), and always used this type of test.  However, it is useful not
because it finds more bugs, but because it does it quicker.

    During the initial stages of debugging a chip, it is convenient to
be able to generate a lot of test vectors quickly, with out spending
much time figuring out which specific parts of the design are being
exercised.  However, long before deciding that a design "works," a
series of comprehensive tests, developed with careful study of both
the specification and the internal workings of the chip, should be
developed and run.

    There is a similar situation with software.  Just because a
program has run correctly a few times with a randomly generated
initial state, DOES NOT mean that it is correct.  In fact, it seems
that such randomization would be a hindrance to locating the source of
a bug.  A single uninitialized pointer bug which corrupts part of
memory is relatively easy to find, IF it always corrupts the same part
of memory.  If there is a random element in the system causing the
uninitialized bug to behave differently with each execution, how can
it be tracked down?  How do you know there's only one bug?  If the bug
doesn't appear until after the program is shipped, how can the user
write a bug report?

    Sorry to get quite so dogmatic about this, but I've seen enough
"finished" products (hardware and software) that broke as soon as they
came near a user that I've gotten a bit paranoid about verification.
In short, I don't really see how hardware/OS modifications that trap
uninitialized variable references would be useful.  Yes, there is
always one more bug, but given a rigorous QA program, the likelyhood
that that bug is an uninitialized variable should be very low.

===============================================================================
Glen Ivey        {ames,uunet,amdahl}!oliveb!cirrusl!glen
Cirrus Logic     glen%cirrusl%oliveb@ames.arc.nasa.gov
--Not only are my opinions not necessarily my companies, they aren't
always mine either.
===============================================================================