🚀 go-pugleaf

What, if any, functional systems available today, which meet most
or all of the following characteristics:

Have a robust/reliable/bet-the-company on it file migration/archival 
system which appears to be a standard unix file system, and which can 
migrate individual files or groups of files to high density secondary
storage.

   -  Transparent handling of online disk, rapid access media such as
      tape cartridges, offline media, and high density, slow access
      media such as optical tape
   -  Programmable migration policy, such as:
         -  Archive on age*size priority
         -  Delay archival of small files
         -  Delay archival of directories
         -  Archive directories of small files together
         -  Prefetch directories of small files together
   -  Scales to millions (billions and billions?) of small files
   -  Supports typical small file access rates scaled appropriately
      to the amount of storage involved
   -  Small files intermixed with very large (Gigabyte+) files
   -  Can support either a global name space, a per-system name space,
      or intermixed
   -  Can support Terabyte+ sized online storage, Petabyte (sp? 10^^15)
      sized offline storage (at least theoretically - access might be slow)
   -  Can support thousands of online and offline secondary storage media
      (e.g. thousands of cartridges)
   -  Fully integrated with multiple available networked file systems,
      especially NFS, AFS/DFS, and "local" protocols (e.g r-commands).
   -  Supports current levels of network security (e.g. Secure NFS,
      DFS security extensions, ACLs, government "orange book"
      Multilevel Security) which current O/S's support
   -  Data rates of up to 10 MBytes/sec supported by the system on
      individual file transfers if the media and networks also support
      such data rates, on both reads and writes, for large files
   -  Intelligent directory handling so that file name lookups are not
      the bottleneck.  Directory scheme not necessary Berkeley FFS based,
      although it could be.
   -  Available on multiple platforms, and scalable, from small servers
      with ~10GBytes online disk storage, to very large systems with
      Terabytes of online disk and Petabytes (sp?) of archival storage
   -  Multilevel hierarchy of archival supported (e.g. archive files on 
      a distributed server to a backup central archival server
   -  Duplication/backup/mirroring available in some form so that
      designated files can be duplicated in two physical locations


OK, I know there is NOTHING out there today which can do ALL of this, but,
how close are commercial products and academic or government projects
to meeting these requirements?

Example:

Cray DMF on a Cray Unicos system might seem to meet robustness, data rate,
and large file handling, but probably can't handle millions or billions
of small files, among other things in the list.

How close do other hardware/software systems come to meeting these
criteria?


-- 
  Hugh LaMaster, M/S 233-9,     UUCP:      ames!lamaster
  NASA Ames Research Center     Internet:  lamaster@ames.arc.nasa.gov
  Moffett Field, CA 94035-1000  Or:        lamaster@george.arc.nasa.gov 
  Phone:  415/604-1056                     #include <usenet/std_disclaimer.h>