🚀 go-pugleaf

ANSI X3B11.1 and ECMA TC15 have been working closely together to 
produce a standard logical file format for optical media (both
read/write and WORM).  ECMA standard 167 is the most recent result of
that effort.

I have posted here a set of frequently asked questions for those
of you interested in knowing more.

Regards,

Ed


------------------------------------------------


Frequently Asked Questions about ECMA 167

What is ECMA 167?

ECMA 167 is a standard published by the European Computer 
Manufacturers Association.  Its full title is Volume and File 
Structure of Write-Once and Rewritable Media Using Non-Sequential 
Recording for Information Interchange.  In short, ECMA 167 describes 
a standard way to write files and directories, along with their 
contents and attributes, on rewritable and write-once optical disks.

Who developed it?

The work was begun nearly four years ago in the ANSI accredited 
task group X3B11.1.  After the completion of the technical work, 
ECMA became interested and adopted the X3B11.1 working paper as a 
standard.  Representatives on X3B11.1 included AT&T, DEC,  Eastman 
Kodak, Hewlett Packard, IBM, Jet Propulsion Labs, Laser Magnetic 
Storage, Micro Design International, Mitre Corporation, Sony, Sun 
Microsystems, Wang Labs, and others.

What problems did they want to solve with ECMA 167?

The basic vision of the committee was to put in place a standard 
that would allow users to exchange data between different operating 
systems and machine architectures as easy as users exchange floppy 
disks between PC's.

Sounds pretty ambitious.

Yes, it is.  But people are doing ambitious things with computers and 
there is no fundamental reason why this capability cannot be 
achieved.  Making a sequential, tape-like format would leave some 
basic issues unaddressed.  For instance, many data sets are so large, 
that it is getting impractical to move them to another medium for 
transport, backup or archive.  It should be possible to process the 
data directly on the exchange medium if necessary.  A format that 
does not preclude a single format solution for processing, backup, 
archive and interchange was the goal.

One of the members (HP) has already demonstrated totally 
transparent mounting of ECMA 167-like media on Macintosh, UNIX 
and DOS systems.  Specifically, files can be written directly to 
optical (WORM or rewritable) by any off-the-shelf application on 
say, a Mac and read by any application that can make sense of the 
file contents (TIFF, GIF, WKS, etc.) on a PC or UNIX system.

You mean I should abandon my own format?

Absolutely not!  Many formats are tuned for specific requirements or 
have such historical precedence that they should continue to be 
supported and enhanced.  In many cases, the addition of a simple 
utility to allow the import and export of media in the ECMA 167 
format would provide the needed compliance.

What are some of the benefits of ECMA 167?

Extensible - First and foremost, ECMA 167 is designed to 
accommodate the varying needs of applications and operating 
systems to represent files and some of their most fundamental 
attributes in an independent way.  ECMA 167 provides a 
hierarchical file system with directories and subdirectories 
much like UNIX.  In addition, 167 attempts to provide generic 
"hooks" intended to allow specific constructs needed by many 
popular operating systems -- like the Macintosh Resource Fork 
for instance.  The architects of 167 did understand that they 
could not anticipate all possible future requirements, and so they 
liberally provided areas for the recording of implementation 
specific data and a standard means for identifying who wrote 
those areas.  The entire format is designed to evolve in a 
controlled fashion as requirements dictate.

Security - The high capacity of modern optical products allow the 
concentration of enormous quantities of valuable data in a single 
piece of media.  File systems that accept this responsibility must 
not allow single point failures and must provide help to recovery 
applications in reconstructing damaged disks.  ECMA 167 uses 
tagged data structures which allow the unambiguous 
identification of unlinked file system structures.  Redundancy is 
allowed in many areas as well.

Large Data Sets - ECMA 167 standardizes an approach for handling 
multi-volume sets that allow individual files to be larger than a 
single piece of media.

Support for International Use - As business becomes increasingly 
global, it is essential to accommodate the needs of those using 
alternate character sets, especially multi-byte approaches such 
as Japanese shift-JIS standard and UNICODE (ISO 10646).  ECMA 
167 completely supports their use in all character fields.

Is ECMA 167 compatible with ISO 9660?

ISO 9660, sometimes called High Sierra, is a logical file format for 
CD-ROM media.  Because of the unique characteristics of CD media 
and because it is a file format intended to be mastered and not 
updated, the architects of ECMA 167 could perceive no value in 
attempting to extend ISO 9660 data structures to accommodate 
WORM and rewritable media.  On the other hand, great value was 
attributed to being able to fully represent the files and attributes 
when the contents of a CD-ROM recorded according to ISO 9660 was 
copied to a medium recorded according to ECMA 167.

Anything else?

There was cooperation with another committee charged with 
upgrading ISO 9660 to use the new Compact Disc Write-Once 
technology being offered by Phillips.  This included coordination of 
basic data types, extended attributes, record types for large 
systems, character sets and the sharing of a common registration 
authority for the identification of implementation use areas.  

In addition, a gamble was taken to encourage the large system 
companies to consider an eventual change to their boot ROM's.  A 
scheme to allow the booting of multiple operating systems from a 
single piece of media that is compatible with format standards for 
CD-ROM, CD-WO, WORM and rewritable media was defined and agreed 
upon.

The format sounds complex.  I just want to get simple interchange of 
files.  And how about performance?

No doubt, building a system capable of originating and receiving 
media recorded with the highest defined levels of ECMA 167 will be 
a distinctly non-trivial task.  Anticipating this, the committee 
defined three levels of compliance for the file system, the lowest 
level severely restricting certain constructs and the highest level 
defining no restrictions.  In between, the idea of conformance 
domains were introduced.  Conformance domains allow a set of 
cooperating originators and receivers to define additional 
restrictions to simplify exchange among themselves.  A group 
interested in using only a subset of ECMA 167 capabilities could 
register a conformance domain which would dictate for instance: 
"Level 2 compliance with only 8 bit ASCII characters, no 
subdirectories and no extended attributes".  This would provide 
considerable simplification for the originators, yet allow any fully 
conforming level 2 receiver to read the media.

As for performance, ECMA 167 specifically supports certain very 
high performance strategies such as file compression, file space 
preallocation, pre-erased sector management (for magneto-optical 
recording) and position independence of format data structures.  
Custom data structures to accelerate file look ups are supported.  
This is a rich area for vendors to explore adding value to their own 
implementations of 167.

How is ECMA 167 structured?

To further simplify conformance, ECMA 167 is divided into 5 
distinct parts: 1) common definitions and data structures, 2) volume 
identification and booting, 3) volume structure and partitioning, 4) 
file structure, and 5) record structure.  

You mean I don't have to use the file format part?

Right, if you have a demonstrated need to use an alternative file 
format, you could use parts 1 and 3 of ECMA 167 to offer standard 
volume structure and partitioning features and still use a file 
format more suited to your requirements.  This approach should be 
done judiciously, because a proliferation of overlapping file formats 
could have an "inflationary" effect, reducing the value of all 
standards.

What standardization efforts remain?

ECMA 167 is a standard today.  It will move into the ISO 
standardization process later in 1992 and probably become an ISO 
standard sometime in 1993.  It appears there is sufficient support 
world wide to allow this process to continue smoothly.  X3B11.1 has 
chosen to pursue some advance features compatible with ECMA 167, 
but not currently found there today.  Those include means for 
identifying the encryption and compression of files and a specific 
methodology to support striping.  ECMA will beginning work soon to 
develop a compatible set of standards for tape.

How do I get more information?

Write to the chair of X3B11.1:

Edward Beshore
Hewlett Packard Company
Greeley Storage Division
700 71st Avenue
Greeley, Colorado 80634