🚀 go-pugleaf

In <9002140907.AA02472@net.bio.net> AKC01@dk0rrzk1.bitnet (Kay Hofmann) says:
> The usage of the program and the algorithms involved should be documented
> well enough to understand what the program is good for.  (This was a real
> trivial remark)

	I agree with Kay, but he doesn't go far enough.  First, by calling
it a trivial remark, he grossly understates the importance of what he said.
Second, he doesn't emphasize enough the difference between a program and an
algorithm.  The former is simply an implementation of the latter.  From the
papers I read, most authors don't seem to understand the difference, and
spend their time describing the program, but never say anything about the
algorithm.  Since you can't write it yourself, you end up writing to the
author and ask for a copy of the code, only to discover that what you get is
10,000 lines of non-documented VAX/VMS fortran with imbedded VMS keyed file
management system calls, TEK-4010 graphics output, and depending on VT-100
escape sequeces for interactive input.  Don't laugh, I've seen it.  Often.

	The rule they try to teach for scientific papers is that there
should be enough detail in the materials and methods so that anybody
reading the paper can reproduce the work.  For a software paper, that means
the algorithm is described in enough detail that anybody should be able to
write a program implementing the algorithm and get the same results.  That
also means that the paper should provide a test data set (reference to a
GenBank entry by ascession number, for example) and the results when that
data set is run through that algorithm so a programmer can test his
implementation.  And just like how a bench scientist need not explain what
he or she means by "phenol extract" or "M13 sequencing", a programmer need
not explain "hash table" or "quicksort".  On the other hand, "The two
sequences are aligned using a dynamic programming algorithm" doesn't begin
to come close to saying enough to let somebody implement it.  Any paper
which describes how you run a program, but not the underlying algorithm,
should be flat-out rejected.  If the person using the program doesn't
understand the algorithm, he's stupid if he believes the results because he
has no basis on which to judge their validity.  If the description is there
and he chooses not to take the effort to understand it, it's his own
stupidity, but he needs to be at least given the chance.  That thought is
important enough that it deserves repeating, and its own paragraph.

	Any paper which describes how you run a program, but not the
underlying algorithm, should be flat-out rejected.  If the person using the
program doesn't understand the algorithm, he's stupid if he believes the
results because he has no basis on which to judge their validity.  If the
description is there and he chooses not to take the effort to understand
it, it's his own stupidity, but he needs to be at least given the chance.

	Given the above, I would say that fully half (if not more) of the
biological software papers I've seen should have been rejected.  This
applies to NAR, CABIOS, Nature, Science, PNAS, etc.
--
Roy Smith, Public Health Research Institute
455 First Avenue, New York, NY 10016
roy@alanine.phri.nyu.edu -OR- {att,philabs,cmcl2,rutgers,hombre}!phri!roy
"My karma ran over my dogma"