🚀 go-pugleaf

In article <457@oracle.UUCP>, bradbury@oracle.UUCP (Robert Bradbury) writes:
> I have a proposed Benchmark Specification (based on DeWitt) from a company
> in Palo Alto, CA (International Data Corporation Technology Laboratories)
> which attempts to address the concurrancy and system impact issues.
> In theory they intend to run it on all of the RDBMS.  The only problem
> is that it requires the dedication of several machines for several
> weeks in order to run it and that is alot of time (money) to run tests which
> are likely to be obsolete in 6 months given the evolution of these systems.
> 
> There is also the problem that all of the RDBMS have different interface
> languages so you have to write a seperate benchmark for each system meaning
> you are to a degree comparing apples with oranges.  Until everyone conforms
> to the ANSI X3H2 SQL standard it will be very difficult to realistically compare
> one RDBMS with another.
> 

While I agree that the query language could have an impact on the 
outcome of a benchmark, I think that its contribution would be
minor compared to other factors. If the RDBMS supports a programmatic
interface (which I feel any serious contender on Un*x should), then
the query language could, to some degree, be factored out,
provided the programmatic interface was not simply an embedding
of the query language.

I also disagree that RDBMS systems are evolving so fast that any benchmarks
will be oboslete in 6 months. The underlying RDBMS technology is now
fairly mature, so mature that various vendors feel comfortable offering
products to the public. In order for a sound technical decision to
be made, an important (and in some applications *critical*) factor
is RDBMS performance. Without solid data about how a particular
RDBMS performs, how can a sound technical decision to buy be made?
A customer who is interested in buying now doesn't want to be told
that (s)he should wait six months, and persumably the company is
also interested in making the sale.

I'm glad to see that a standard is beginning to take shape. Perhaps
the best solution would be to have some third party (like BYTE or
UNIX WORLD) do the benchmarks and publish the results. This has
typically been how benchmarking has been done for C and Pascal
compilers.


		Jim Kempf	hplabs!kempf


<<<<<**** usual disclaimer ****>>>>>