🚀 go-pugleaf

Submitted-by: Christophe.Wolfhugel@univ-lyon1.fr (Christophe Wolfhugel)

david@mks.com (David Rowley) writing about PAX:
> larger field sizes for dates, times, file lengths, etc.

And also, something that nor tar nor cpio have: large file names.

The current limit of around 100-128 characters for the filename
is catastrophic. Simply too short!

Also I guess users would appreciate: the ability to archive
in absolute format and restore in relative. Not all users do
relative archives. And one knows how dangerous an absolute
archive can be. Even when I get a tape from the oustide I prefer
being able to restore it in relative mode, without having to
check before it all files on the archive are or not absolute.

-- 
Christophe Wolfhugel    |    Email: Christophe.Wolfhugel@univ-lyon1.fr
   "No keybord, press F1 to continue"


Volume-Number: Volume 28, Number 60

Submitted-by: henry@zoo.toronto.edu (Henry Spencer)

>Submitted-by: Christophe.Wolfhugel@univ-lyon1.fr (Christophe Wolfhugel)
>The current limit of around 100-128 characters for the filename
>is catastrophic. Simply too short!

Actually, the 1003.1 tar format accommodates 255.  Still a bit tight
for some people.

>Also I guess users would appreciate: the ability to archive
>in absolute format and restore in relative...

This is really a user-interface issue, not a format issue.  There's no
reason why tar couldn't have had some sort of renaming facility from
the start (and we certainly are long overdue to have it added).
-- 
There is nothing wrong with making      | Henry Spencer @ U of Toronto Zoology
mistakes, but... make *new* ones. -D.Sim|  henry@zoo.toronto.edu  utzoo!henry


Volume-Number: Volume 28, Number 61

Submitted-by: djm@eng.umd.edu (David J. MacKenzie)

   And also, something that nor tar nor cpio have: large file names.

   The current limit of around 100-128 characters for the filename
   is catastrophic. Simply too short!

POSIX.1 cpio format has no limit on the length of pathnames, although
some existing implementations may place artificial limits on the
length.  POSIX.1 tar format has a limit of 100 to 256 bytes, depending
on where slashes are located in the path.

   Also I guess users would appreciate: the ability to archive
   in absolute format and restore in relative. Not all users do
   relative archives. And one knows how dangerous an absolute
   archive can be. Even when I get a tape from the oustide I prefer
   being able to restore it in relative mode, without having to
   check before it all files on the archive are or not absolute.

As GNU tar has demonstrated, the handling of leading slashes in
pathnames is independent of the archive format.  It's a trivial bit or
two of code in the archiving program that can be used with any format.


Volume-Number: Volume 28, Number 62

Submitted-by: david@mks.com (David Rowley)

Actually, if you want more info on joining POSIX working groups, etc
contact:

	Lisa Granoien
	IEEE Computer Society
	1730 Massachusetts Ave NW
	Washington, DC 20036-1903
	(202) 371-0101
	FAX: (202) 728-9614

David.

-- 
David Rowley
Mortice Kern Systems Inc.
35 King Street North, Waterloo, ON, Canada N2J 2W9
519/884-2251, FAX 519/884-8861, david@mks.com


Volume-Number: Volume 28, Number 63

Submitted-by: peter@ferranti.com (Peter da Silva)

I'm not familiar with the ISO1001 format, but I would like to suggest a textual
self-documenting format, like RFC822. You don't need to put the actual field
names in every file header if you establish a property file at the beginning of
the archive. For example:

	FieldCodeset: ASCII
	FieldSeparator: <0A>

[ these two would be required for the initial header, and need to be in an
  agreed on language. ]

	HeaderLength: 96
	BodyLength: 0
	Common: HeaderLength,BodyLength,FileName,ModificationTime,Owner

[ This establishes the common fields until the next Common field ]

	2746
	13995
	usr/local/src/pax/Makefile
	19920723.194516.0001
	peter

[ These are the header length, body length, filename, etcetera for the first
  file ]

	FileComment: Remember to edit this for local architecture[...]

[ This is a part of the file system on the originating system. Would probably
  end up in usr/local/src/pax/.Unresolved/Makefile on UNIX ]

	#!/bin/make
	[...]

This should require a minimal amount of additional space, and would be
indefinitely extensible. Older versions of the archive program could
discard unknown headers, or stick them in a file somewhere for later
messing around with. It should also be relatively easy to treat as a fixed
record length format.

> 	- Support for >32 bit systems

Inherent.

> 	- Be extensible, to allow for future support of security information,
> 	  contiguous files, system management information, special files,
> 	  larger field sizes for dates, times, file lengths, etc.

Inherent.

> 	- Support portability of file, user, group names and other
> 	  file information across machines of differing codesets and/or
> 	  locales (i.e. it should work on ASCII, EBCDIC, ISO 10646 and
> 	  Shift-JIS systems alike).

With the caveat that a standard character set be used for the first two fields.

> 	- The archive must be able to be stored on as many media types
> 	  as possible, including but not limited to 9-track and streaming 
> 	  tape, diskette, CDROM, ftp files, etc.

Inherent.

> 		- Encryption of individual files in the archive

Simple extension: new file header.

> 		- Compression of individual files in the archive

Ditto.

> 		- Translation of file contents between differing
> 		  codesets (i.e. format only needs to handle translation
> 		  of file *names*, etc)

Ditto. ( add Body-Type )

> 		- Storage of filesystem specific information (such as inodes)

Ditto.

> 		- Direct mailability of the archive file (without translation,
> 		  such as through uuencode)

Probably ditto, assuming no magic in the bodies or lines that are too long
for mail.

This is obviously not a complete proposal. I would appreciate feedback and
suggestions, and if they seem positive I'll go ahead and write a real proposal
up. The intellectual property status and usage of RFC822 is presumably not
going to be a problem.

-- 
Peter da Silva                                               `-_-'
$ EDIT/TECO LOVE                                              'U` 
%TECO-W-OLDJOKE Not war?                        Have you hugged your wolf today?
Ferranti Intl. Ctls. Corp.      Sugar Land, TX  77487-5012       +1 713 274 5180


Volume-Number: Volume 28, Number 64

Submitted-by: les@chinet.chi.il.us (Leslie Mikesell)

In article <152l4tINNist@ftp.UU.NET> peter@ferranti.com (Peter da Silva) writes:
>Submitted-by: peter@ferranti.com (Peter da Silva)

>I'm not familiar with the ISO1001 format, but I would like to suggest a textual
>self-documenting format, like RFC822. You don't need to put the actual field
>names in every file header if you establish a property file at the beginning of
>the archive. For example:
>	FieldCodeset: ASCII
>	FieldSeparator: <0A>
>[ these two would be required for the initial header, and need to be in an
>  agreed on language. ]

What's a field?  And why so verbose in something that will only be
parsed by programs or people who better know what they are doing?
How about short abbreviations for the common header items?

>	HeaderLength: 96
>	BodyLength: 0
>	Common: HeaderLength,BodyLength,FileName,ModificationTime,Owner
>[ This establishes the common fields until the next Common field ]
>	2746
>	13995
>	usr/local/src/pax/Makefile
>	19920723.194516.0001
>	peter
>[ These are the header length, body length, filename, etcetera for the first
>  file ]

What's the advantage of this over explicit token/value pairs for each header
item?  If you don't repeat the token for each item, how will you recover
from errors where the initial description is lost?  

I agree with the concept though.  We should only need one format for
encapsulating any attributes that any OS might need to carry around
if it is done with header records that can be added as necessary.
Textual components can be noted as such for conversion as necessary
to extract on any other system.  Other data types can be identified
to the extent possible and the type of encoding/encryption/compression
can be noted per-item.  Using a mailable encoding should produce a
mailable archive (i.e. the header fields should be textual). 

>This should require a minimal amount of additional space, and would be
>indefinitely extensible. Older versions of the archive program could
>discard unknown headers, or stick them in a file somewhere for later
>messing around with.

Not only older versions will have to deal with this but versions under
different OS's that don't have the same storage concepts. 

>> 	- The archive must be able to be stored on as many media types
>> 	  as possible, including but not limited to 9-track and streaming 
>> 	  tape, diskette, CDROM, ftp files, etc.

One thing that is needed here is to be able to generate multi-part
archives, possibly spliting files across media, with the ability
to extract files from any single part directly.   

>> 		- Compression of individual files in the archive

Yes, and the compression should restart as you go to new media to allow
the above splitting to work, or the header should note that the file
can only be extracted as a continuation.

>> 		- Direct mailability of the archive file (without translation,
>> 		  such as through uuencode)

This should be a possibility by using an appropriate encoding but not
enforced for backup media.   For backup purposes the archiver should
be able to note the current contents of directories and which of those
items are included in the archive (like GNU tar with the -G option).
This allows restoring incremental backups without accumulating the
files that were deleted between the runs.

Les Mikesell
  les@chinet.chi.il.us

Volume-Number: Volume 28, Number 65

Submitted-by: henry@zoo.toronto.edu (Henry Spencer)

>Submitted-by: les@chinet.chi.il.us (Leslie Mikesell)
>[822 as archive format]... Other data types can be identified
>to the extent possible and the type of encoding/encryption/compression
>can be noted per-item...
>One thing that is needed here is to be able to generate multi-part
>archives, possibly spliting files across media...

One thing that disturbs me about this suggestion is that the participants
don't seem to be aware of RFC 1341, which extends RFC 822 in a number of
ways that are relevant to this discussion.  RFC 1342 is also noteworthy.
-- 
There is nothing wrong with making      | Henry Spencer @ U of Toronto Zoology
mistakes, but... make *new* ones. -D.Sim|  henry@zoo.toronto.edu  utzoo!henry


Volume-Number: Volume 28, Number 66

Submitted-by: peter@ferranti.com (Peter da Silva)

In article <155ha9INNlrq@ftp.UU.NET> les@chinet.chi.il.us (Leslie Mikesell) writes:
> What's a field?

A "line", "entry", ... (shut up, Data).

Also, someone noted in Email that a third entry for block size (so you could
pad the headers and data blocks for easy DMA-ing if you wanted to) would be
desirable.

> And why so verbose in something that will only be
> parsed by programs or people who better know what they are doing?

Good point. The final form would obviously want to be shorter. I wouldn't
want to fix the size of tags or anything, or make them unintelligable.
With the "common" headers the size of tags is pretty unimportant.

Another comment in Email was that the "common" fields should be delimited
and left as prefixes, so you could so something like:

	Common:@FileName:/usr/peter/tosave@

And leave off common sections of filenames and things.

> >[ These are the header length, body length, filename, etcetera for the first
> >  file ]

> What's the advantage of this over explicit token/value pairs for each header
> item?

Space saving.

> If you don't repeat the token for each item, how will you recover
> from errors where the initial description is lost?  

Good point. You could periodically repeat the common description, perhaps,
or if you're really unsure about the media just leave it off altogether. Or
you could try and figure it out from context... one of the points to this
format is that you would have a reasonable chance of doing that.

> >This should require a minimal amount of additional space, and would be
> >indefinitely extensible. Older versions of the archive program could
> >discard unknown headers, or stick them in a file somewhere for later
> >messing around with.

> Not only older versions will have to deal with this but versions under
> different OS's that don't have the same storage concepts. 

Yes, I think I referenced that in my example with the FileComment field.
Shoudl definitely be made explicit, though.

> 
> >> 	- The archive must be able to be stored on as many media types
> >> 	  as possible, including but not limited to 9-track and streaming 
> >> 	  tape, diskette, CDROM, ftp files, etc.
> 
> One thing that is needed here is to be able to generate multi-part
> archives, possibly spliting files across media, with the ability
> to extract files from any single part directly.   

That's a good point. How about this:

	FileName: /usr/lib/news/history
	Segment: 2/6
	Offset: 10485532

> Yes, and the compression should restart as you go to new media to allow
> the above splitting to work, or the header should note that the file
> can only be extracted as a continuation.

Good.

> This should be a possibility by using an appropriate encoding but not
> enforced for backup media. 

This could be a compression option:

	Compression: UUENCODE

(expansion, really. :->) If it offends, leave it implicit.

> For backup purposes the archiver should
> be able to note the current contents of directories and which of those
> items are included in the archive (like GNU tar with the -G option).
> This allows restoring incremental backups without accumulating the
> files that were deleted between the runs.

Good.
-- 
Peter da Silva                                               `-_-'
$ EDIT/TECO LOVE                                              'U` 
%TECO-W-OLDJOKE Not war?                        Have you hugged your wolf today?
Ferranti Intl. Ctls. Corp.      Sugar Land, TX  77487-5012       +1 713 274 5180


Volume-Number: Volume 28, Number 69

Submitted-by: hcbarth@afterlife.uucp (Bart Bartholomew)

	I once used a system that had disk sectors as the
basic storage unit.  Obviously not Unix.  At the end of each
record (sector) on the backup medium, there were a few extra
pieces of information:
	64-bit checksum (Not mod2 add, a rather more complex
			calculation.)
	current filename
	current recordnumber of filename
	number of records in filename
	Administrative trivia, like owner, etc.
The unusual point here is the explicit support for multi-volume
archives.  In particular, you could split a file over two tapes,
*restore from tape 2 first* (it was sometimes convenient), and
then from tape1.  The numrec of numberofrecs information came
in handy many many times.  In fact, sometimes we would have
two drives going at the same tim during restore, and properly
reconstruct the files.  On a full restore, we had eight drives
going simultaneously at full speed.  A nice part of the system
was that we could reconstruct the directories starting at any
tape in the set.  If lables got lost, no big deal.
	Clearly, support for multiple volumes is needed.
	Bart
-- 
"It's not the thing you fling, the fling's the thing." - Chris Stevens
 If there's one thing I just can't stand, it's intolerance.
*No One* is responsible for my views, I'm a committee. Please do not
infer that which I do not imply.  hcbarth@afterlife.ncsc.mil


Volume-Number: Volume 28, Number 74