🚀 go-pugleaf

kre@munnari.OZ.AU (Robert Elz) writes:
> [ .. ]
>It would truly be nice if everyone could avoid using ftp to fetch
>files from the US - at the very least, look on archie.au first.
>What's more, if a file is not actually there in the regular mirrors,
>you can always have archie.au fetch it for you, and cache the result so
>should anyone else want the same file it doesn't need to be transferred
>over the US link again.

How does one do this? (Have something cahed at archie.au)

>The first 30 minutes of operation of the new link saw more than 160Mb
>imported into Aust (total bytes, includes ftp & everything else)
>now after about 50 minutes we're at about 280 Mb.
>Just where is all that data going???

Into porno scrapbooks?

Regards,
	-Matt

kre@munnari.OZ.AU (Robert Elz) writes:

>The link to the US was out of action for about 30 minutes this
>morning while it was upgraded from 512 kb/sec to 768 kb/sec.

>Within minutes (actually, within seconds) the link was already dropping
>packets on ping turnaround tests (and no, it wasn't due to line problems
>anywhere)

>We're already back to having a saturated link - I've seen peak link
>usages of 750kb/sec (that's a 5 min weighted average - obviously any
>time a packet is being transmitted the actual instantaneous usage
>will be 768kb/sec, which is not a useful number to know), with usage
>currently sitting (well) above 740kb/sec consistently.
[...]
>The first 30 minutes of operation of the new link saw more than 160Mb
>imported into Aust (total bytes, includes ftp & everything else)
>now after about 50 minutes we're at about 280 Mb.

>Just where is all that data going???

In an attempt to find out where it's going, could a similar breakup to the
NSF traffic reports be done on the USA-AU link ?

They manage to come out monthly (albiet a bit late) and give a breakdown
of network traffic both a port basis (percentages and 'real' figures).

jatz already has reports on link usage for each state available, but these
don't give much clue as to what is going across the satellite.  Is a per
state breakdown (even per net ?) of traffic possible of traffic going
to/from the USA ?

While its good to see the satellite doing its work, its also interesting
that the load seemed to increase as students commenced classes again.

Darren

p.s. does anyone know why the link between Melbourne and Canberra has been
     having short periods (upto 30 seconds) of packet loss ?  Last night
     ping was returning time exceeded from the gateways and occasionally
     unreachable.

In <matth.732433404@extro.ucc.su.OZ.AU>
matth@extro.ucc.su.OZ.AU (Matthew Hannigan) writes:

| kre@munnari.OZ.AU (Robert Elz) writes:
| >What's more, if a file is not actually there in the regular mirrors,
| >you can always have archie.au fetch it for you, and cache the result so
| >should anyone else want the same file it doesn't need to be transferred
| >over the US link again.

| How does one do this? (Have something cahed at archie.au)

I did mean to actually explain in my message, and its been pointed out
several times that I should have ...

First, note that this stuff is experimental, and is not guaranteed to
always be working.

The mechanism is to look in /alex on archie.au - in there you will
find directories named after top level domains, inside each (most)
of those you'll find directories named after sub-domains, and so on
until you find directories named after hosts.  (Caution: some domains
are hosts, which contain files, but which also have sub-domains).
Note that just doing "ls" doesn't necessarily show everything available
at this level, only hosts and domains that have either been referenced
before, or have been pre-wired into the cache.   You can just try a "cd"
to a path name that doesn't exist to add new domains or directories
(assuming that the host or domain actually exists).   Also note that
brousing around here can be very slow, you're always almost much better
off simply taking the host name that you want to access, reversing the
domains (ala JANET), and replacing all dots ('.') with slashes ('/')
so as to make a normal looking unix path-name, then cd to that - directly
to the host name in one step.   (eg: "cd /alex/au/oz/munnari" if you
wanted to fetch files from munnari.oz.au - however please avoid fetching
files from inside Australia this way, conserve cache space for foreign
based files - also check that archie.au doesn't have a copy of the file
in one of its normal mirror areas, and fetch from there if it does).

Once you're in the host directory, you're effectively at the root of
that host's anonymous ftp area, and you can cd down into it just as
if you had ftp'd to that host - if the directories already exist on
archie.au this will be quick, if they don't, archie.au will need to
ftp to the host you're accessing to obtain the details, which can make
operations that seem as if they should be quick take a bit longer than
you might expect (but just once).

You fetch files in the normal ftp way - alex will check to see that the
file isn't out of date if it has a copy in its cache, or it will fetch
the file to its cache if it wasn't there - also note that this will cause
a delay as (I think) the file won't start to be sent back to you until
the whole thing has reached archie.au, so for big files you can expect
a lengthy delay after "get'until bytes start appearing - but once they
do they'll be coming just from archie.au, so should be much faster than
fetching from overseas.

I've never tried it, but I rather doubt that this will work if the
destination ftp site isn't a (normal kind of) unix server.

Again, please note, this is experimental, its possible that it won't
always be available at all (if /alex is empty, the alex server isn't
running on archie.au) and its also possible that it won't always work
perfectly.  Alex also has limits on how many ftp's it will attempt
simultaneously, if that limit is exceeded, later ftp attempts will
simply hang.

kre

kre@cs.mu.oz.au (Robert Elz) writes:

>| kre@munnari.OZ.AU (Robert Elz) writes:
>| >What's more, if a file is not actually there in the regular mirrors,
>| >you can always have archie.au fetch it for you, and cache the result so
>| >should anyone else want the same file it doesn't need to be transferred
>| >over the US link again.

>| How does one do this? (Have something cahed at archie.au)

>I did mean to actually explain in my message, and its been pointed out
>several times that I should have ...

>First, note that this stuff is experimental, and is not guaranteed to
>always be working.

One thing I did notice, when I was checking out the incoming
directory of Sunsite.unc.edu, I was simultainusly using
regular FTP to sunsite, and trying out Alex to Sunsite as well..

What I found was, that on alex, the incoming was the incoming of
a few days ago, whereas on the REAL ftp, the files had long ago been moved
and new files were there.

So, is this the cache not reading the new directory? or some glitch?
-- 
Anthony Rumble
aka SmilieZ
"Anything is possible.. If there is enuf money in it"

Anyone who has ncftp installed on their Unix box (and I recommend
you do), might like the following script which uses the alex 
filesystem/cache to fetch files from overseas.

I call this alex, and when I want to get something, I tend to say

	alex some.site.overseas:/some/path/name/to/ftp &

Sure, it takes a little longer, but who really cares how long FTP 
takes?

Written by Michael Henry...

#!/usr/local/bin/perl

$alexhost="archie.au";

foreach $arg (@ARGV) {
	if ( $arg =~ /:/ || $arg =~ /.+\..+/ ) {
		($host,$path) = split(':', $arg);

		$host = join('/', reverse split(/\./, $host));

		if ( ($path ne "") && ($path !~ m,^/,) ) {
			$path = "/$path";
		}

		$arg = "$alexhost:/alex/$host$path";
	}
}

unshift(@ARGV,"ncftp");
exec(@ARGV);

In article <1oke00INN43m@rastaban.cc.deakin.OZ.AU>, ccw@deakin.edu.au
(Craig Warren) wrote:
> 
> Anyone who has ncftp installed on their Unix box (and I recommend
> you do), might like the following script which uses the alex 
> filesystem/cache to fetch files from overseas.
> 
> I call this alex, and when I want to get something, I tend to say
> 
> 	alex some.site.overseas:/some/path/name/to/ftp &
> 
> Sure, it takes a little longer, but who really cares how long FTP 
> takes?
> 
> Written by Michael Henry...

Excellent script!  And it uses perl too :-).  Seriously though, everyone
should install and try out this script, it works like a charm - now for a
Mac version :-)
   Peter.

_______________________________________________________________________
Peter N Lewis <peter@cujo.curtin.edu.au>             Ph: +61 9 368 2055

ccw@deakin.edu.au (Craig Warren) writes:

>Anyone who has ncftp installed on their Unix box (and I recommend
>you do), might like the following script which uses the alex 
>filesystem/cache to fetch files from overseas.

What I have done (experimentally) is set up an Alex file system on one of
my hosts and then modified the Hostlist file so that all the top level
domain references are actually pointers to /alex/au/archie/alex. 
ie /alex/edu -> /alex/au/archie/alex/edu

This means that all file access overseas via Alex so through the 
cache on archie.au. It does work, but since I only set it up yesterday 
I'm not sure what practical problems having two levels of Alex
causes. 

The Alex server talks the NFS protocol so a single server can be
accessed from multiple machines. It is a very nice interface to
the anonymous ftp world.

I did have to delete a line of code from the server which prevented
going through multiple servers.
--
------------------------------------------------------------------
 Toivo Pedaste                        Email:  toivo@ucs.uwa.edu.au
 University Computing Services,       Phone:  +61 9 380 2605
 University of Western Australia      Fax:    +61 9 380 1014

I run MHSnet and when I want to get something, I say

	netfetch some.overseas.site /path/file

:-)

In article <1993Mar24.050105.1603@cs.su.oz.au>, bob@cs.su.oz.au (Bob Kummerfeld) writes:
> I run MHSnet and when I want to get something, I say
> 
> 	netfetch some.overseas.site /path/file
> 
> :-)
> 

As another alternative / enhancement, would it be possible to enhace the
wonderfull ftpmail@cs.uow.edu.au to include an optional command to force
it to use alex,

eg

ALEX archie.au
HOST some.overseas.site
GET /path/file

Bruce Powell			email: bss_brucep@vd.seqeb.gov.au