🚀 go-pugleaf

kre@cs.mu.oz.au (Robert Elz) writes:
>In <48pk2c$2ih$1@haywire.DIALix.COM>
>peter@haywire.DIALix.COM (Peter Wemm) writes:

>|  8  border4-fddi-0.SanFrancisco.mci.net (204.70.3.163)  7.649 ms  7.532 ms  13.458 ms
>|  9  aarnet.SanFrancisco.mci.net (204.70.35.6)  196.438 ms  188.199 ms  191.951 ms
>|     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>| 10  * national-aix-us.gw.au (139.130.29.1)  207.283 ms  203.449 ms

>| Note that all traffic is going via San Francisco/Melbourne
>| (national-aix-us.gw.au is a name for the USA interface on national.gw.au)

>There is no US interface on the national gateway ("aix" is someone's idea
>of a label for part of the hub ethernet in Melb - Australian Internet
>eXchange).

Yep. Even I knew better when I thought about it..  I vaguely recall
being told how it was set up eons ago now that I think about it...  My
brain must have been out to lunch... :-(  From memory (and from some
corrections I got in email over the last few days :-), national.gw.au
is the hub, and usa.gw.au is another router altogether connected to
national with ethernet and the interface "name" on usa.gw.au happens
to have a .mci.net dns name (which is what threw me), and which one
shows up on traceroute is an artifact of which interface the packet
arrives on when the TTL has hit zero and the forwarding rejects it.
Also, wasn't clix.aarnet.edu.au(?) sniffing the national<->usa.gw.au
ethernet for packet counting?


>| The other odd bit is that the delay going from the mci core border to
>| the aarnet router in SanFrancisco is incurring a large hit.. Could
>| that be flooded too perhaps?

>Huh?   "aarnet.SanFrancisco.mci.net" a name for the US serial
>link interface on the box called "usa.gw.au" - that is in Australia.

>Even if you didn't know that, and there's no real reason you should,
>consider the delats, 180 ms (approx) from border4 to aarnet... and
>then another few ms (5-10) from aarnet... to national-aix.   Which of
>those is likely to bethe cross pacific link?   Really!

Yeah, it was brain overload.. My mistake.  I've seen traceroute do
funny things, and returning abnormally high numbers for a hop was one
of them..  I dismissed that as a traceroute bogon rather than the real
trans pacific hop.. :-/

>This is a good chance to remind everyone that DNS labels provide a way to
>convert human usable, and relatively stable, identifiers into the relatively
>meaningless (except to IP routing) and human unfriendly IP addresses
>that are requried to transport data.   Reading anything more than "this name
>was assigned by xxx" into a DNS name is usually a mistake.   They don't
>have the meanings that people like to attribute to them.

>| I suspect that TIS are going to have to bite the bullet and split the
>| australian bgp "area" into two parts...

>This would enable symmetric routing, for what that is worth (something,
>but not necessarily a lot).   It would also mean that which link you are
>using for incoming traffic (which is the only direction that really matters,
>the outgoing links aren't congested) would depend entirely upon who you are
>(where your site is connected), whereas now it depends on who you are talking
>to (where the other site is connected).   If one of the links is better
>than the other that would means some Aust sites would always get the "good"
>one, and others always the "bad" one.   The current scheme is probably better,
>regardless of issues of routing table sizes in the US.

>kre

I just did a couple of pings across both of the links, about a minute
apart (and verified them both again to make sure it was reasonably
consistant).

(this was about 6:30am sydney time, and 3:30am perth time)
Script started on Thu Nov 23 11:36:04 1995

peter@freefall.cdrom.com[11:38am]~-101> ping -c 50 aarnet.sanfrancisco.mci.net
PING aarnet.sanfrancisco.mci.net (204.70.35.6): 56 data bytes
64 bytes from 204.70.35.6: icmp_seq=2 ttl=246 time=331.720 ms
64 bytes from 204.70.35.6: icmp_seq=3 ttl=246 time=388.367 ms
64 bytes from 204.70.35.6: icmp_seq=5 ttl=246 time=305.678 ms
64 bytes from 204.70.35.6: icmp_seq=6 ttl=246 time=354.591 ms
64 bytes from 204.70.35.6: icmp_seq=7 ttl=246 time=191.427 ms
64 bytes from 204.70.35.6: icmp_seq=8 ttl=246 time=191.228 ms
64 bytes from 204.70.35.6: icmp_seq=14 ttl=246 time=384.722 ms
64 bytes from 204.70.35.6: icmp_seq=16 ttl=246 time=332.763 ms
[...]
64 bytes from 204.70.35.6: icmp_seq=37 ttl=246 time=371.123 ms
64 bytes from 204.70.35.6: icmp_seq=43 ttl=246 time=225.783 ms
64 bytes from 204.70.35.6: icmp_seq=44 ttl=246 time=198.977 ms
64 bytes from 204.70.35.6: icmp_seq=45 ttl=246 time=212.475 ms
64 bytes from 204.70.35.6: icmp_seq=46 ttl=246 time=194.571 ms
64 bytes from 204.70.35.6: icmp_seq=48 ttl=246 time=217.562 ms
64 bytes from 204.70.35.6: icmp_seq=49 ttl=246 time=215.508 ms

--- aarnet.sanfrancisco.mci.net ping statistics ---
50 packets transmitted, 30 packets received, 40% packet loss
round-trip min/avg/max = 189.790/282.321/459.616 ms

peter@freefall.cdrom.com[11:40am]~-102> ping -c 50 aarnet-telstra.losangeles.mci.net
PING 204.70.171.66 (204.70.171.66): 56 data bytes
64 bytes from 204.70.171.66: icmp_seq=0 ttl=245 time=712.255 ms
64 bytes from 204.70.171.66: icmp_seq=1 ttl=245 time=713.925 ms
64 bytes from 204.70.171.66: icmp_seq=2 ttl=245 time=710.270 ms
64 bytes from 204.70.171.66: icmp_seq=3 ttl=245 time=711.109 ms
[....]
64 bytes from 204.70.171.66: icmp_seq=46 ttl=245 time=726.819 ms
64 bytes from 204.70.171.66: icmp_seq=47 ttl=245 time=703.312 ms
64 bytes from 204.70.171.66: icmp_seq=48 ttl=245 time=711.656 ms
64 bytes from 204.70.171.66: icmp_seq=49 ttl=245 time=702.720 ms

--- aarnet-telstra.losangeles.mci.net ping statistics ---
50 packets transmitted, 50 packets received, 0% packet loss
round-trip min/avg/max = 702.720/714.268/792.826 ms
peter@freefall.cdrom.com[11:43am]~-103>

Script done on Thu Nov 23 11:43:46 1995
(full version on http://haywire.dialix.oz.au/~peter/usalinks.txt)

Even at this early time of day, the 6Meg link was dropping 40% of it's
packets on the way from the USA to Melbourne.  At the same time, the
LA->Sydney link dropped nothing, and was pleasantly stable on it's RTT
time.

What I'm wondering, is if the USA and Europe are on the 6Meg link,
what's *left* of the rest of the Internet to go on the other one?
About the only thing I could think of was Japan and other
Asian countries, and perhaps south America and Africa.  Aside from
Japan, the leftovers strike me as not likely to be particularly strong
on traffic with Australia.  I could be wrong, I dont know the
numbers... My guess would be that 90% of the traffic coming into *.au
would be originating in the US or Europe...  (Does anybody have real
numbers handy?)

I do some work over a telnet to the USA..  I'm slowly going (further)
around the twist waiting for the TCP exponential backoff to recover
from the excessive dropped packets...  (so much so that I've
contemplated writing a brute force IP tunneling widged based on
simple positive ack UDP encapsulation and aggressive non-backoff
retransmission.. It's not as hard as it sounds as most kernels these
days have a user mode IP tunnel... And those that dont usually have
at lease slip and pseudo ttys.) 

-Peter