🚀 go-pugleaf

I noticed this morning that hostnames in CHI.IL.US could not be
resolved (IP address and MX records couldn't be looked up by the
world at large) and that the problem seemed to have existed for more
than 12 hours.  I found that the two nameservers for CHI.IL.US,
although attached to different local providers (XNet and MCS) were
fed through Sprintlink, and Sprintlink had a big problem.

Thanks to wonderful cooperation from Paul Southworth of the regional
provider CICNET and Shanthi Ranganathan of the US Domain Registry at
ISI, I was able to get a third nameserver online and added to the
delegation list the same day.  (The CICNET server was able to get the
zone information from the primary on XNet through the Chicago NAP,
bypassing Sprintlink's internal partition.)  Mail soon began to flow
again to CHI.IL.US destinations which didn't directly depend on the
failed connection.

I extend big thanks to Paul and Shanthi for their rapid response.
And of course it was no fault of the local providers who host the
other two nameservers that they were both taken out of contact with
the world by a single point of failure.
__________________________________________________________
Matt Crawford                      matt@severian.chi.il.us
ISOC & IETF member; 15-year internaut; concerned e-citizen