🚀 go-pugleaf

In article <EMV.91May22165647@poe.aa.ox.com> emv@msen.com (Ed Vielmetti) writes:

>If you can specify exactly where the files you are interested in are,
>it is straightforward to maintain a reasonably up to date shadow
>directory of the remote FTP site and make that directory available for
>UUCP.  Updates can be handled in a number of ways

[...]

>Naturally you'd like to keep whatever mechanisms there are for making
>sure that the file you fetch is the same as the latest version on the
>server somewhat hidden from the general "rftp" user, so that your
>caching will have a reasonable chance to work.  No sense in going off
>to bother Columbia if you have a copy handy that's the same as what
>they have.

If the file changes frequently and is not requested often, caching would
just be a waste.  Besides, bitftp at Princeton was able to scarf a
500K file from Columbia within a second of asking, according to the
FTP log they send back - is it really worth storing a copy to save
that second?  On the other hand, it is pretty inconvenient for me to
determine when that file has been changed.

What you really need is a new transfer protocol that tells you up front 
the date and length of a file and a CRC value for the contents.  If
you have a matching copy, then you don't need to do the transfer at
all.  If your copy is shorter, you would send a CRC and the length of
the piece you have and the other end would check it against the corresponding
length of its copy.  If the CRC matched, you could just pick up the
new portion of the file, otherwise the old copy would be completely
overwritten.   Ideally, this protocol would work through one or more
"relay" sites that would be allowed to cache locally, passing on only
the initial check to the remote end point.

Les Mikesell
  les@chinet.chi.il.us