🚀 go-pugleaf

RetroBBS NetNews Server

Inspired by RockSolid Light RIP Retro Guy

Article View: comp.mail.mush
Article #1004

Re: Compressed folders

#1004
From: Barton E. Schaefer
Date: Fri, 02 Feb 1990 01:50
159 lines
7047 bytes
(There ought to be an easier way to include the text of three articles
than aborting followup on the first two and then reading in dead.article
during followup to the third ... ah, well ....)

In article <3233@taux01.UUCP> crehta@taux01.nsc.com (Ran Ever-Hadani) writes:
} I have a few folders which are large enough, and accessed rarely
} enough that I keep them compressed.  How hard should it be to have mush
} recognize file names ending with .Z in the folder directory
} to be compressed, zcat them into its temporary file, and re-compress
} when a save is issued to that folder?

In article <9002011752.AA00414@kalli.corp.sun.com> kevin@Corp.Sun.COM (Kevin Sheehan {Consulting Poster Child}) writes:
} 
} Yep - this is definitely the case for me too.   Is also way down on
} my plate of things to do.  From what I can see it's easy enough for
} folders, since you don't have to worry about anyone else accessing
} them at the same time.

I don't know what "way down on my plate" means, but let me make some
remarks about the difficulties with such a scheme.  When loading a
folder, mush parses the message headers, computes message sizes, and
copies to the temp file in a *single* pass; the folder is not copied
first and then parsed.  This makes it somewhat difficult to interpose
a process like zcat.  We hope eventually to be able to load a folder
from standard input or a pipe, but parsing some folder formats requires
seeking, so there's a lot of work involved.

Similarly, on update, mush makes one pass through the tempfile copying
only those messages not marked as deleted.  It would be easier to
introduce a pipe through compress at the update end of things.  However,
if the "update" command is used rather than "folder" or "quit", mush
re-parses the updated folder to re-create the temp file -- which would
necessitate uncompressing again, and make "update" pretty slow.

"save" to a compressed folder would be agonizing; uncompress, append the
message, and compress again.  A better plan would be to save messages to
an ordinary folder with a related name, and occasionally gather together
all the messages by uncompressing the primary folder, "merge"with the 
secondary folder, and then update and compress the whole thing and remove
the secondary folder.

In article <1990Feb1.194232.12005@usenet.ins.cwru.edu> trier@SCL.CWRU.Edu (Stephen Trier) writes:
} Wouldn't it be possible to implement folders in the way you mentioned by
} using a command alias?  I haven't mastered the use of such things yet, but
} it should certainly be possible
} 
} Of course, using this technique would keep all of your folders compressed
} unless you gave the cmd a new name.  (How about zfolder, zfo for short?)

This is probably the best solution, which I would have proposed if
Stephen hadn't beaten me to it. :-)  I'd suggest making a mush script
that is called from a cmd alias, though, so "if" expressions can be
used to detect compressed folders.  (One of these days the regular
parser will be able to handle "if" constructs!)  The new syntax for
variables in 7.0 can be really helpful here.  In fact, here's a shot
at the scripts.  They've even been tested a little.

Add to .mushrc:

    # usage: zfolder [compressed-folder [temp-file-name]]
    cmd zfolder "set z='Z \!*'; source +.zfolder"	# Z is placeholder
    cmd zupdate "source +.zupdate"			# ignores arguments
    cmd zfo zfolder
    cmd zup zupdate

In file .zfolder in your $folder directory:

    # Mush script to use compressed folders
    # Needs $z set to "Z [arg1] [arg2]", see zfolder cmd
    # Bug: can't handle folder names beginning with + or ~
    #  (use file completion: zfo +^[name where ^[ is escape)

    if "X$z:2" != X			# if z is set to something
	if "X$z:2" =~ "X*.Z"		# if it has the .Z
	    set newfolder = $z:2	# use the name given
	    set had_Z			# and remember that
	else				# otherwise
	    set newfolder = ${z:2}.Z	# put the .Z on
	    if $?had_Z
		unset had_Z		# and forget it
	    endif
	endif
    else				# otherwise
	if $?compress
	    echo -n '"'"$compress"'"'" loaded as "
	endif
	folder				# act like folder with no args
	exit
    endif

    # this is just flashiness to demonstrate using multiple arguments
    if "X$z:3" != X			# if there's a second argument
	set tmpfolder = $z:3		# use it as the uncompressed file
    else				# otherwise
	set tmpfolder = /usr/tmp/zfo$$	# conjure up a file name
    endif

    # now we get to the hard part
    if -e $newfolder			# if the compressed file exists
	set zfo				# flag value
	source +.zupdate		# update the current folder
	echo -n "uncompressing $newfolder ... "
	sh zcat $newfolder > $tmpfolder	# uncompress the new folder
	echo "loading"
	if $?ok
	    unset ok			# remove ok if set
	endif
	folder $tmpfolder | set ok	# ok set if folder succeeds
	if $?ok
	    headers			# display headers
	    set compress = $newfolder	# remember name of original
	endif
	unset zfo newfolder tmpfolder	# clean up
    else				# otherwise
	if $?had_Z			# if the arg had .Z
	    echo "$newfolder not found"	# complain about it
	    unset had_Z
	else				# otherwise
	    set ok zfo			# flag value
	    source +.zupdate		# update the current folder
	    unset ok zfo
	    folder $z:2 | set ok	# try an ordinary folder
	    if $?ok
		headers			# display headers
	    endif
	endif
    endif

In .zupdate in your $folder directory:

    # Mush script to update a compressed folder

    set oldfo = $thisfolder		# save current folder name
    if $?zfo				# if called from .zfolder
	folder /dev/null		# sneaky update without reparse
    else				# otherwise
	update				# normal update
    endif
    if $?compress			# if this folder was compressed
	echo -n "compressing ... "
	if $?zfo			# if called from .zfolder
	    sh compress $oldfo		# compress it again
	    echo -n "updating $compress ... "
	    sh cp $oldfo.Z $compress	# update the compressed version
	    sh rm $oldfo.Z		# get rid of the temp version
	    unset compress		# clean up
	else				# otherwise
	    sh compress < $oldfo > $compress	# don't alter $oldfo file
	endif
	echo "done"
    endif
    unset oldfo				# clean up

Just having more fun than Randal Schwartz,
-- 
Bart Schaefer                             "Live and don't learn, that's us."
                                                                   -- Hobbes

schaefer@cse.ogi.edu (used to be cse.ogc.edu)
Technical Details
Message-ID: <7053@ogicse.ogc.edu>
Path: arc1-ca.pugleaf.net!utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!ucsd!ogicse!schaefer
References: <3233@taux01.UUCP> <9002011752.AA00414@kalli.corp.sun.com> <1990Feb1.194232.12005@usenet.ins.cwru.edu>

Show Full Headers
Complete Article Headers
Path: arc1-ca.pugleaf.net!utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!tut.cis.ohio-state.edu!ucsd!ogicse!schaefer
From: schaefer@ogicse.ogc.edu (Barton E. Schaefer)
Newsgroups: comp.mail.mush
Subject: Re: Compressed folders
Message-ID: <7053@ogicse.ogc.edu>
Date: 2 Feb 90 01:50:14 GMT
References: <3233@taux01.UUCP> <9002011752.AA00414@kalli.corp.sun.com> <1990Feb1.194232.12005@usenet.ins.cwru.edu>
Reply-To: schaefer@ogicse.UUCP (Barton E. Schaefer)
Organization: Oregon Graduate Institute (formerly OGC), Beaverton, OR
Lines: 159