🚀 go-pugleaf

In article <46078@ogicse.ogi.edu> nabil@ogicse.ogi.edu (Aaron Nabil) writes:
>I'd like to write a freely availiable archie server, preferably one
>that can read existing archie databases.
>
>If you'd be interested in help out in planning or implementation, 
>drop me a line.

Well, as one of the guys who has devoted most of the past
couple of years of his life to building up and keeping the
existing archie service alive I would have to point out
that the code _is_ freely available. Of course, it is no
longer _free_ to an archie service operator, which is a
different issue, but those who buy it also get a lot of
support, including coordination between the multiple
archie sites, bug fixes, answers to a _lot_ of user's
questions, documentation, etc. I say this only in case
anyone who might be wondering about why we must now charge
for access to archie wonder exactly what we are charging
for....

Although I certainly don't want to discourage
experimentation, I think anyone thinking of turning an
archie-like tool loose on the net should consider the
impact that hundreds or thousands of similar programs
gathering the same data would have on volunteer information
providers, the problems of bandwidth drain at your site (a
popular service can have a _significant_ effect on your
resources) and even the problems providing user support
for freebie services that require significant admin
support. Note that these are all operational issues - we
found that the coding was the easy bit. Supporting the
_service_ (not the project) was the hard bit.

Having said all of this, if you would like to know how
_we_ did it, you might want to pick up a copy of the paper
we did on archie for the January, 1992 Usenix conference.
I believe that it is available at most of the archie sites
through anonymous FTP. It has been circulating for a
while, so should be easy to find (I'd like to be able to
point to a specific machine at McGill, but sadly the
McGill volunteer machine for archie died and noone has
been willing to help the net out with a replacement.
Anyone with suggestions on how we could get such a public
service machine back online in a world where no money is
changing hands for use of what seems to be a useful and
popular service are most welcome to post ideas...)

BTW, I'm not sure I understand what you mean by "read
existing archie databases"? The new release of archie is
capable of supporting arbitrary database systems and
ships with a copy of the public domain version of WAIS.
If you are referring to the ability of querying an
existing archie server for its data, then simply pick up
any of the freely available archie clients and look at the
Prospero code that talks to archie. It's available in
source and is very well written (by Cliff Neuman at ISI).

If you are referring to the anonymous FTP collecions that
we actually track, well simply login and do a dir command.
If you are referring to the ability to read an already
built archie database available directly on your machine,
well this would presumably only be of interest to people
operating our version of archie. Sure the database format
we provide for the anonymous FTP archie database has a
programatic interface but we have not made this freely
available since only those operating an archie server can
even access the database in such a format, so there is no
need to release this (or am I mising something here?)

There is one other possibility you might be referring to
that I can see. We have mechanisms for ensuring
consistency between multiple archies and you might
presumably wish to sync your "archie" (and personally, I'd
really wish you picked another name for what will
obviously be a different project. The people working on an
archie-like server for Gopher called theirs "veronica" and
this certainly should be appreciated by users who could
only be confused by unnecessary naming collisions like
this).

Unfortunately, as a struggling commercial start-up we
don't have the resources to subsidize such an effort by
providing information on this syncing mechanism, but if any
of our clients are interested in this they should give us
a call (after all, they're paying for consultation
support... :-) 

If it turns out that a global database syncing mechanism
is something for which there is significant demand, then
work should probably be done to standardize this through
the IETF and if such an effort is started we'd be glad to
participate (in fact, I'm at the IETF as I write this). We
are certainly not averse to working to make Internet
services interoperable and the IETF is the forum for this.



				- peterd

In article <46078@ogicse.ogi.edu> nabil@ogicse.ogi.edu (Aaron Nabil) writes:
>I'd like to write a freely availiable archie server, preferably one
>that can read existing archie databases.
>
>If you'd be interested in help out in planning or implementation, 
>drop me a line.

Well, as one of the guys who has devoted most of the past
couple of years of his life to building up and keeping the
existing archie service alive I would have to point out
that the code _is_ freely available. Of course, it is no
longer _free_ to an archie service operator, which is a
different issue, but those who buy it also get a lot of
support, including coordination between the multiple
archie sites, bug fixes, answers to a _lot_ of user's
questions, documentation, etc. I say this only in case
anyone who might be wondering about why we must now charge
for access to archie wonder exactly what we are charging
for....

Although I certainly don't want to discourage
experimentation, I think anyone thinking of turning an
archie-like tool loose on the net should consider the
impact that hundreds or thousands of similar programs
gathering the same data would have on volunteer information
providers, the problems of bandwidth drain at your site (a
popular service can have a _significant_ effect on your
resources) and even the problems providing user support
for freebie services that require significant admin
support. Note that these are all operational issues - we
found that the coding was the easy bit. Supporting the
_service_ (not the project) was the hard bit.

Having said all of this, if you would like to know how
_we_ did it, you might want to pick up a copy of the paper
we did on archie for the January, 1992 Usenix conference.
I believe that it is available at most of the archie sites
through anonymous FTP. It has been circulating for a
while, so should be easy to find (I'd like to be able to
point to a specific machine at McGill, but sadly the
McGill volunteer machine for archie died and noone has
been willing to help the net out with a replacement.
Anyone with suggestions on how we could get such a public
service machine back online in a world where no money is
changing hands for use of what seems to be a useful and
popular service are most welcome to post ideas...)

BTW, I'm not sure I understand what you mean by "read
existing archie databases"? The new release of archie is
capable of supporting arbitrary database systems and
ships with a copy of the public domain version of WAIS.
If you are referring to the ability of querying an
existing archie server for its data, then simply pick up
any of the freely available archie clients and look at the
Prospero code that talks to archie. It's available in
source and is very well written (by Cliff Neuman at ISI).

If you are referring to the anonymous FTP collecions that
we actually track, well simply login and do a dir command.
If you are referring to the ability to read an already
built archie database available directly on your machine,
well this would presumably only be of interest to people
operating our version of archie. Sure the database format
we provide for the anonymous FTP archie database has a
programatic interface but we have not made this freely
available since only those operating an archie server can
even access the database in such a format, so there is no
need to release this (or am I mising something here?)

There is one other possibility you might be referring to
that I can see. We have mechanisms for ensuring
consistency between multiple archies and you might
presumably wish to sync your "archie" (and personally, I'd
really wish you picked another name for what will
obviously be a different project. The people working on an
archie-like server for Gopher called theirs "veronica" and
this certainly should be appreciated by users who could
only be confused by unnecessary naming collisions like
this).

Unfortunately, as a struggling commercial start-up we
don't have the resources to subsidize such an effort by
providing information on this syncing mechanism, but if any
of our clients are interested in this they should give us
a call (after all, they're paying for consultation
support... :-) 

If it turns out that a global database syncing mechanism
is something for which there is significant demand, then
work should probably be done to standardize this through
the IETF and if such an effort is started we'd be glad to
participate (in fact, I'm at the IETF as I write this). We
are certainly not averse to working to make Internet
services interoperable and the IETF is the forum for this.



				- peterd

peterd@cc.mcgill.ca (Peter Deutsch) writes:
+In article <46078@ogicse.ogi.edu> nabil@ogicse.ogi.edu (Aaron Nabil) writes:
+>I'd like to write a freely availiable archie server, preferably one
+>that can read existing archie databases.
+>
+>If you'd be interested in help out in planning or implementation, 
+>drop me a line.
+
+>Well, as one of the guys who has devoted most of the past
+>couple of years of his life to building up and keeping the
+>existing archie service alive I would have to point out
+>that the code _is_ freely available.

Freely availiable, as in _free_.  Or at least availiable.

>Of course, it is no
>longer _free_ to an archie service operator, which is a
>different issue, but those who buy it also get a lot of
>support, including coordination between the multiple
>archie sites, bug fixes, answers to a _lot_ of user's
>questions, documentation, etc. I say this only in case
>anyone who might be wondering about why we must now charge
>for access to archie wonder exactly what we are charging
>for....

You could try answering your email when commercial
sites like mine with money in their fists write you asking how
to set up an archie server. 

>... I think anyone thinking of turning an
>archie-like tool loose on the net should consider the
>impact that hundreds or thousands of similar programs
>gathering the same data would have on volunteer information
>providers,...

No doubt, but that's why I want to be able to read existing archie
databases.

>BTW, I'm not sure I understand what you mean by "read
>existing archie databases"?

For example, ans.net has their archie database publicly readable.  Several
archie sites are mirroring it for their own use.


-- 
Aaron Nabil
nabil@cse.ogi.edu  

If you've got the time and resources to develop a new AFTP database
server, why wouldn't you consider contributing that time and effort
towards improving the existing one?

If you feel that Alan and Peter have done the net a great disfavour by
charging for the new releases of Archie servers, perhaps you could
elaborate. I personally accept what they did as being necessary to
sustain the development of the service(s). I also don't think that
they're in it to bilk the net out of something, or to make a fortune
on the backs of others. I think their contributions to IETF and such
activities reflects a genuine concern for and interest in sustained
growth of the Internet community. The fact that they also have to eat
and pay rent without the benefit of graduate stipends or a job that
allows them to work on their own projects is simply a fact of life.

My money is where my mouth is: I spend more time than I should
"supporting" xarchie and developing the next version, which will be
released sometime RSN, maybe this month. I'm happy to contribute for
free to the extent that my situation allows it. They wanted to be
independent of external factors like theses or backups, and so choose
the path they did. The fact is that the Internet is becoming
commercialized at an incredible rate, and I think it's great that
knowledgeable and caring people like Alan and Peter are involved. It's
one thing to write a bit of code that sets up the servers -- we can
all do that. It's another thing entirely to devote time to community
concerns that such services generate.

George
-- 
George Ferguson                 ARPA: ferguson@cs.rochester.edu
Dept. of Computer Science       UUCP: rutgers!rochester!ferguson
University of Rochester         VOX:  (716) 275-2527
Rochester  NY  14627-0226       FAX:  (716) 461-2018

I have sent Aaron a separate piece of email with more
detail, but have a couple of comments for the whole group.

In article <46131@ogicse.ogi.edu>
nabil@ogicse.ogi.edu (Aaron Nabil) writes:
>peterd@cc.mcgill.ca (Peter Deutsch) writes:
>+In article <46078@ogicse.ogi.edu> nabil@ogicse.ogi.edu (Aaron Nabil) writes:
>+>I'd like to write a freely availiable archie server, preferably one
>+>that can read existing archie databases.
.  .  .
>
>You could try answering your email when commercial
>sites like mine with money in their fists write you asking how
>to set up an archie server. 

Your email apparently got lost, and I apologise for this,
but archie 3.0 is definitely available and now shipping.
Without trying to make excuses, there are a couple of
reasons why this might have happened. Of course, we handle
_lots_ of user-type queries each day so sometimes other
queries get lost in the noise since Alan and Ieach
sometimes assume the other guy answered a particular
letter.  There is currently only Alan working full-time on
the archie project, with me doing more "building of
business" kind of work, plus follow-on projects with our
one employee, so we're spread pretty thin. If ever we
don't answer email for any reason, please don't give up -
try again.

>>... I think anyone thinking of turning an
>>archie-like tool loose on the net should consider the
>>impact that hundreds or thousands of similar programs
>>gathering the same data would have on volunteer information
>>providers,...
>
>No doubt, but that's why I want to be able to read existing archie
>databases.
>
>>BTW, I'm not sure I understand what you mean by "read
>>existing archie databases"?
>
>For example, ans.net has their archie database publicly readable.  Several
>archie sites are mirroring it for their own use.

I now understand - you're talking about the compressed
ls-lR files in the archie listings directories.

We obviosly haven't released any of our parsers (which
filter out error messages and verify that these listings
files are consistent) since they are a core part of the
archie code, and non-trivial programs to get right (the
lack of such a parser for VMS was due to the time it took
us as volunteers to write such a beast). There is such a
parser for the current release and thus VMS host support
will appear as the existing servers convert to the new
code.

Of course, we can't compete with free or subsidized
programming time, but since this code (and providing for
its associated support) is our only source of income we
unfortunately can't release our code to the world. (yes, I
know of the GNU model, but I believe that releasing source
would merely make it easier for someone else to go into
competition with us charging for the support, with us
giving away the last year of our work to set them up in
business. I'm not really convinced that this would be fair
to us).

Anyways, we're still happy to share our experience,
information services model, documentation, etc. We are
currently planning a paper on the architecture of archie
3.0 (once we get a quiet minute or two to look at it) so
those wishing to do something similar can see what we did.
We're also working on several RFCs on information
templates, a quick and dirty guide to Internet tools, etc.
Copies of this sort of material in available from the
machine "archives.cc.mcgill.ca" in the directories under
"pub".



					- peterd