🚀 go-pugleaf

In message <57p80g$qiu@news.ibm.net.il> - ifaa1@ibm.net.il (Yossi Cohen)
writes:
:>
:>ISPs in Europe or Asia pay, as far as I know, about $250K/Year 
:>for a T1 line to the U.S. (for the line only, not for the internet 
:>connection).
:>
:>On the other hand, traffic could have been reduced by a significant 
:>factor by using a large cache in the proxy http and ftp server. I know
:>
:>that the academic network in the UK has a cache of 6 machines with 
:>16GB each and they reach 60% hit rate.
:>
:>It is my guess that by using a huge cache, say 1TB (1000GB), it is 
:>possible to get 90% hit rate, i.e. to save 90% of the international 
:>lines cost. The cost of 1TB may be about $200k, so it seems a very 
:>good investment.
:>
:>Where am I wrong ?   
:>

Well $200K is a tad on the low side if you expect to keep the thing
spinning.  Lots of extra hardware needed to spin that much disk on a single
host.  I'ld say you would be darn lucky to do it for twice that price.

:>Does anyone know of some research done in this field of knowing 
:>how large should a cache be in order to reach 90% hit rate ?
:>

How would you measure it?  It all depends on where the users want to web
surf.  After a while the maint cost on the 1 TB drive array starts to make
the T1 link costs across the pond look pretty cheap.  Your 1 TB array is
going to eat a lot of clean power and need a lot of air conditioning too. 
All these costs have to be paid for.  Would pay many orders of magnitude for
the service?

96GB of web cache may be effective I don't think 1 TB has the same
effectiveness nor do I think you could reach 90% hit rate on a mixed set of
ISP users.  after you cache the poular sites I think folks usage would be
all over the planet.  

//=========================================//
Paul M Blais - pblais@visi.com - St Paul, MN

pblais@visi.com (Paul M Blais) wrote:

>In message <57p80g$qiu@news.ibm.net.il> - ifaa1@ibm.net.il (Yossi Cohen)
>writes:
>:>

>How would you measure it?  It all depends on where the users want to web

One way to measure it, is by using a "virtual cache" of 1 TB, at some
big ISP's proxy server. This virtual cache is actually a list of URLs
(and their size) that are being dropped from the normal cache. Using
such a list may give exact indication on cache size and hit ratio.


>//=========================================//
>Paul M Blais - pblais@visi.com - St Paul, MN

In message <57t0pu$126e@news.ibm.net.il> - ifaa1@ibm.net.il (Yossi
Cohen)Sat, 30 Nov 1996 23:31:23 GMT writes:
:>
:>pblais@visi.com (Paul M Blais) wrote:
:>
:>>In message <57p80g$qiu@news.ibm.net.il> - ifaa1@ibm.net.il (Yossi Cohen)
:>>writes:
:>>:>
:>
:>>How would you measure it?  It all depends on where the users want to web
:>
:>One way to measure it, is by using a "virtual cache" of 1 TB, at some
:>big ISP's proxy server. This virtual cache is actually a list of URLs
:>(and their size) that are being dropped from the normal cache. Using
:>such a list may give exact indication on cache size and hit ratio.
:>

Using that approach I guess you could figure the cost benefit of the 1 TB of
disk and how long it will take to pay for it (or actually how long it will
take subscribers to pay for it).  With the low Inet rates here ($19 / month)
I don't think many ISP's are going to jump at the chance to test your
theory.  There is more money to be made leasing the space than caching the
web.


//=========================================//
Paul M Blais - pblais@visi.com - St Paul, MN

	All the details espace me at this moment, but there was a tiny
article in WIRED magazine a while back that showed one of the foudners
(?) of the Thinking Machines company and his newest venture:  tera
byte storage units.  He was quoted as saying that the web would fit
quite comfortably in one of his units.  


RB

Yossi Cohen wrote:
> 
> ISPs in Europe or Asia pay, as far as I know, about $250K/Year
> for a T1 line to the U.S. (for the line only, not for the internet
> connection).
> 
> On the other hand, traffic could have been reduced by a significant
> factor by using a large cache in the proxy http and ftp server. I know
> 
> that the academic network in the UK has a cache of 6 machines with
> 16GB each and they reach 60% hit rate.
> 
> It is my guess that by using a huge cache, say 1TB (1000GB), it is
> possible to get 90% hit rate, i.e. to save 90% of the international
> lines cost. The cost of 1TB may be about $200k, so it seems a very
> good investment.
> 
> Where am I wrong ?
> 
> Does anyone know of some research done in this field of knowing
> how large should a cache be in order to reach 90% hit rate ?
> 
> Thanks,
> Yossi
> 
> ifaa@aquanet.co.il

Proxy would be great for speed up , but it might be good
for censorship, if it is mandatory.

Beware.

----
Yoel

In article <32A5C332.56D103BC@shani.net>, YOELK <jkrup@shani.net> writes:
>Yossi Cohen wrote:
>> 
>> ISPs in Europe or Asia pay, as far as I know, about $250K/Year
>> for a T1 line to the U.S. (for the line only, not for the internet
>> connection).
>> 
>> On the other hand, traffic could have been reduced by a significant
>> factor by using a large cache in the proxy http and ftp server. I know
>> 
>> that the academic network in the UK has a cache of 6 machines with
>> 16GB each and they reach 60% hit rate.
>> 
>> It is my guess that by using a huge cache, say 1TB (1000GB), it is
>> possible to get 90% hit rate, i.e. to save 90% of the international
>> lines cost. The cost of 1TB may be about $200k, so it seems a very
>> good investment.
>> 
>> Where am I wrong ?
>> 
>> Does anyone know of some research done in this field of knowing
>> how large should a cache be in order to reach 90% hit rate ?
>> 
>> Thanks,
>> Yossi
>> 
>> ifaa@aquanet.co.il
>
>Proxy would be great for speed up , but it might be good
>for censorship, if it is mandatory.
>
>Beware.
>
>----
>Yoel
>
Now, this can be done also by using some "filtering" of requests. No sweat.
So yes, this is true with caching, but this can be done with other means 
as well

There is another problem: Actuality!

You will always get a copy from the past, NOT the current version, from a chache.
HOW outdated a page is in the chache, depends on it's updating policy, but 
it by definition MUST be outdated.

If you are actually talking to a hidden cache, not the real address, this 
IMHO is a clear case of denial of service. If you're talking to a chache 
(ie proxy server) knowingly, this is ok, but there MUST be a way to 
enforce a direct connection to the real source of information.
(recently I had to do this to get to the current version of an 
phone/address directory after I had found out that the cached version 
gives me a outdated number, just a very simple example)


Apart from this: For MOST stuff, handled in a responsible way, caching 
has real virtues and should be considered in a reasonable and 
responsible way.

Uwe

In article <57p80g$qiu@news.ibm.net.il> ifaa1@ibm.net.il (Yossi Cohen) writes:
   ISPs in Europe or Asia pay, as far as I know, about $250K/Year for
   a T1 line to the U.S. (for the line only, not for the internet
   connection).

   On the other hand, traffic could have been reduced by a significant
   factor by using a large cache in the proxy http and ftp server. I
   know that the academic network in the UK has a cache of 6 machines
   with 16GB each and they reach 60% hit rate.

   It is my guess that by using a huge cache, say 1TB (1000GB), it is
   possible to get 90% hit rate, i.e. to save 90% of the international
   lines cost. The cost of 1TB may be about $200k, so it seems a very
   good investment.

   Where am I wrong ?   

I expect that if they ran a smarter protocol across the water, it
would gain them alot right there.  HTTP is _not_ a very smart protocol
for WAN access.  One page might initiate ten seperate TCP connections,
each of which requires seperate setup/teardown overhead, and the
sliding window will always start out small, so large objects will
require many round trips.

Instead, it would make sense to run HTTP to the local proxy, Hyper-G
or other single-connection protocol from the local proxy across the
water, and then convert back to HTTP on this side.  Basically, you'd
multiplex the individual requests into a single large pipe which stays
connected all the time.  That right there would probably save a goodly
chunk of bandwidth and decrease the wall clock time required to
download data, all without any cache-update concerns.

If you did run a cache, I'd want something like Harvest, which I think
can arrange to keep things in sync.  You'd want the US end of the
proxy to keep track what's in the European side of the proxy, and then
send notices to invalidate pages as it sees them change.

Sigh.  It's pretty amazing that the network works as it is.  Who says
that Anarchy _can't_ work :-).

Later,
--
scott hess <shess@one.net>     (606) 578-0412      http://w3.one.net/~shess/
<I plan to become so famous that people buy tapes of me reading source code>

I smiled on 03 Dec 1996 18:59:10 -0500 when R. I. Bullock <rbullock@csxt.cs.oberlin.edu> wrote:


:R I B>	All the details espace me at this moment, but there was a tiny
:R I B>article in WIRED magazine a while back that showed one of the foudners
:R I B>(?) of the Thinking Machines company and his newest venture:  tera
:R I B>byte storage units.  He was quoted as saying that the web would fit
:R I B>quite comfortably in one of his units.  


Um, that would mean that the web consists of only about 1000 1-gig sites,
or 100 10-gig sites. I think it is much bigger than that.


--
+----------------------->Guy Kenneth a.k.a Mr Smiley<-----------------------+
UA SEDS President 96-97 <        .     .            > C/C++ Java Perl Tcl/tK
student/webmaster/sysop >          .     .          < HTML CGI VRML POV UNIX
621-5904 Steward Rm 315 <                 *         > astronomy astrophysics
smiley@seds.org         > "Electrons For Breakfast" < planetary sciences
+=======================<http://www.seds.org/~smiley>=======================+
" Keep on jumping around in a violent manner so as to transfer as 
  much kinetic energy to the person next to you."

> Yossi Cohen wrote:
> > 
> > ISPs in Europe or Asia pay, as far as I know, about $250K/Year
> > for a T1 line to the U.S. (for the line only, not for the internet
> > connection).
> > 
> > On the other hand, traffic could have been reduced by a significant
> > factor by using a large cache in the proxy http and ftp server. I know
> > 
> > that the academic network in the UK has a cache of 6 machines with
> > 16GB each and they reach 60% hit rate.
> > 
> > It is my guess that by using a huge cache, say 1TB (1000GB), it is
> > possible to get 90% hit rate, i.e. to save 90% of the international
> > lines cost. The cost of 1TB may be about $200k, so it seems a very
> > good investment.
> > 
> > Where am I wrong ?
> > 
> > Does anyone know of some research done in this field of knowing
> > how large should a cache be in order to reach 90% hit rate ?
> > 
> > Thanks,
> > Yossi
> > 
> > ifaa@aquanet.co.il
> 



But how do you keep the terabyte cache updated without that T1 line to
the U.S.?  Where do you get to save money?  Someplace, there's still
going to be a trans-atlantic/pacific connection...

RB