🚀 go-pugleaf

Here are the results of the Big Dataset Survey I queried the network
about several weeks ago.  You may remember that I asked the question
"What is the biggest dataset YOU have ever pushed through AVS?" The
numbers are interesting for several reasons: the sizes of the datasets
presented, and the variety of techniques for which AVS is being used.

Here is a summary table followed by the brief descriptions everyone
sent me.  Please continue to send me this information - I will
continue to compile it and present the results at the User's
Group Meeting in May! 

Fields:
	67,108,864 Bytes - Marc Kessler, U of Mich. Hospital
			 - (Anonymous)
	60,000,000 Bytes - Ronald J.G. Goossens, Stanford University
	51,800,000 Bytes - Wade K. Smith, Convex Computer Corporation
	24,576,000 Bytes - Jamie Pelagatti, Harvard Medical School
	20,961,952 Bytes - Ron Karwoski, BIR, Mayo Foundation
	12,582,912 Bytes - Larry Gelberg, Advanced Visual Systems Inc.
	 8,388,608 Bytes - Pierre MARTEL, C N E T PAA/TIM/SSI          
	 3,000,000 Bytes - Rob Lake, BP Research
	   120,000 Bytes - Andrew Rhodes, MIT

UCD: 
	   262,144 cells - Alexander Yarmarkovich, Advanced Visual Systems Inc.
	   100,000 cells - Peter Yerburgh, Intera
	    16,405 cells - John Sheehan, Advanced Visual Systems Inc.

----------------------------------------------------------------------

67,108,864 Bytes
	256x256x128 doubles - 3D convolutions
	Stellar GS with 128M
	Marc Kessler, University of Michigan Hospital

----------------------------------------------------------------------

67,108,864 Bytes 

	I've done volume rendering of a 512x256x128 uniform integer volume 
	on a XXXXX with XXXXX, and my volume renderer runs 
	in about XXXXX seconds.  At this point the results are too
	preliminary to be publishable (since I just wrote the renderer).
	But I'll let you know when the results are publishable 
	(probably about the time we release the module).

	XXXXX, XXXXX (wishes to remain anonymous until results are
	officially published)
----------------------------------------------------------------------
60,000,000 Bytes

We have a 3D semiconductor simulation code that models the electrical
behavior of submicron transistors. It solves the Poisson equation and
two transport equations. The output consists of electrical potential,
electron and hole concentration, electron and hole current density at
each node in the computational mesh, i.e. 9 double-precision
floats/grid point. The code is called STRIDE and uses rectangular
blocks as mesh elements.

We are involved in a number of high-performance computing efforts and
have been running STRIDE on an in-house Intel hypercube. Early 1992 we
got access to the Intel DELTA machine at Caltech in Pasadena, CA.
There, we were able to analyse a full series of electrical biases (13
different solutions) on a grid of 5 million points (approximately
150x150x210). Any scalar solution variable represented 20Mb per bias
point, vector variables are 60Mb per bias point. The total solution
file contained 13x9x4x5M bytes = 2.34 Gbyte.

These data were transferred to the National Center for Computational
Electronics at the University of Illinois. The most interesting visual
corresponded to a 3D streamline rendition of hole current for one of
the bias points, with a color contour map of the magnitude of hole
current for the central plane. That data set in itself contained 60
Mbyte of data.

I think - but am not sure - that the visualization was done on a
Convex machine with 2Gbyte of main memory. It is one of Convex's
high-end versions. The streamline visualization took about 16 hours of
CPU time. The visualization was done "distributed" in the sense that
we had hooked up to display windows to the geometry viewer, one
locally at Illinois, the other at Stanford.

The simulation and visualization received an honorable mention in the
Intel Grand Challenge Award, which was organized last year to
commemorate the first anniversary of the DELTA machine.

Ronald J.G. Goossens			
Integrated Circuits Lab, Stanford University

----------------------------------------------------------------------

51,800,000 Bytes
	UCI cell data set (courtesy of D.B. Menzel, et. al.)
	360x360x100 float field (51.8Mb) converted to byte (12.9Mb)
	Convex C220 (2 heads)
	512Mb physical memory
	gradient shade 144 seconds
	tracer 14 seconds
	isosurface plus render (grew to 800Mb) 30 minutes
	work done by Duane Gustavus at Convex

	Wade K. Smith, Convex Computer Corporation

----------------------------------------------------------------------

24,576,000 Bytes
	We have sent 320x320x60 integer (i.e 4 byte) data sets through
	AVS networks. These are CT data sets typically. We've used a
	Stardent GS2000 with 128 Mb of memory. 	Usually, we read it
	then slice it or byte-reduce it before rendering it.  Just
	reading it into a field did not take more than a few minutes.
	It will also flow through the pipes with a couple of minutes too.

	Jamie Pelagatti, Harvard Medical School

----------------------------------------------------------------------

20,961,952 Bytes
	I have loaded a 20 Mb file, 226 x 272 x 341, (byte field) on a
	HP 9000/720 with 64 Mb of memory. The data is a CT scan of the 
	head which we interpolated to make cubic. After the initial
	reading in of the data I am able to render and display the
	data in ~5 seconds. Render types include: Voxel Gradient
	Shading, Maximum Intensity Projections, and Summed
	Projections. Threshold values and three rotation angles (x, y,
	z) can be set by the user. 

	Read ANALYZE->ANALYZE Render->Colorizer->Display Image

	Read ANALYZE and ANALYZE Render are our own AVS Modules which
	have limited ANALYZE functionality. 

	I'm sure we can go larger, I don't have any larger datasets handy
	to find the limit.


	Ron Karwoski                   Internet:   rak@bru.mayo.edu
	Biomedical Imaging Resource, Mayo Foundation
----------------------------------------------------------------------

12,582,912 Bytes
	256 x 256 x 192 byte volume
	Volume Rendering using TRACER and COMP SHADE
	Stellar GS 2000 w/ 128 MBytes 
	
	Larry Gelberg, Advanced Visual Systems
----------------------------------------------------------------------

8,388,608 Bytes
	It takes near 3 minutes for a 3D float field with color 524288
	particules with a "scatter dot" module.

524,288 Bytes
	It takes near 2 secondes for a 2D float field 128x1024 throught
	the "field to mesh" module.

500,000 Bytes
	It takes near 5 secondes for a 3D float field 50x50x50 with an
	"isosurface" module.

	and, of course, all these modules are connected out a "render geometry".
	I'm using AVS on a Stardent 800 with 32 Mbytes of memory.

	Pierre MARTEL, 	C N E T   PAA/TIM/SSI          

----------------------------------------------------------------------

3,000,000 Bytes  (1000 * 250 * 4 (sizeof float) * 3 ([XYZ]))
 	Field 2-D, some 3-D  Fields up to 1000 by 250 or so.  Now using
        3-D topo map about 175 by 150, rendered with 608 by 865 image.
	KPC Titan 3000VS dualprocessor - 128MB
	At 60,000 to 100,000 triangles in the generated geometry, rotation
	and scaling are at the limit of usability.  3-D topo map quite
	usable interactively. 

	Rob Lake, BP Research

----------------------------------------------------------------------

120,000 bytes (10000 * 4 (sizeof float) * 3 ([XYZ])
	We were displaying surfaces (as geometries) comprised of 10,000 points
	at a pretty good speed.  For reasons I won't get into, we actually created
	text files N x 10000 lines long with N surfaces in them.  The module that
	reads the files used a direct positioning which worked nearly instantly.
	We did find that jumping up to 15,000 points crossed a threshold and led
	to big slowdowns -- likely we ran out of memory and it had to swap.

	Our system is a DECstation 5000/200 with Ultrix 4.2, DECwindows, and AVS 4.0.
	The system has 40 Meg RAM, 160 Meg swap space. 

	Andrew Rhodes, MIT

----------------------------------------------------------------------

UCD:

262,144 cells   Alexander Yarmarkovich, Advanced Visual Systems Inc.
		(hydrogen.dat converted to cells for testing purposes)
100,000 cells	Peter Yerburgh, Intera
 16,405 cells   John Sheehan, Advanced Visual Systems Inc.

----------------------------------------------------------------------


--
=== Larry Gelberg ============================ larryg@avs.com =======
      Advanced Visual Systems Inc. (AVS Inc.)
      300 Fifth Ave, Waltham, MA 02154
===== Tel: 617-890-4300 = Fax: 617-890-8287 =========================