Re: Newshistory checking

Article View: demon.ip.developers

Article #74

Back to demon.ip.developers Previous Next

#74

From: Terrance Richard Boyes
Date: Thu, 15 Jul 1999 00:00

28 lines
2492 bytes

Terrance Richard Boyes <tez@pierrot.co.uk> wrote:
> Dave {Reply Address in.sig} <noone@llondel.demon.co.uk> wrote:

>> On the subject of KA9Q, will a hash table (as used in KA9Q) do the

> I didn't even consider KA9Q, and having grabbed the code, yep
> it uses a nice simple hashing algorithm. Now all I've got to
> do is work the numbers out so it'll work nicely for 64K entries
> *in* RAM rather than 64KB *of* RAM !

"porting" the deduping stuff was a doddle, more like intelligent
cut&paste :)

And it's fast, fast enough that I've only just got a big enough
newshistory to give it a reasonable test for how fast it sets
the hashtable up. Even on the 68060 (50MHz) it can hash > 16000 
messageids per second. On the deduping side it manages > 1000
checks/second (whether they're dupes or not).

And if my calcs are correct that means the NewsBorg would have
to be fully filling a 256KB compressed line to make the deduping
a bottleneck.

Thanks all.

-- 
<URL:http://www.pierrot.co.uk/>                                  Team AMIGA
The Jackal only eats bad food.

Technical Details

Message-ID: <19990715.002916.0AA80202.Uaaa@pierrot.co.uk>

Path: arc1-ca.pugleaf.net!unknown!not-for-mail

References:

<19990713.200500.0AA80202.PEaa@pierrot.co.uk> <slrn7oo9bq.j5g.chris@ccserver.keris.org> <abbaryybaqryqrzbapbhx.fevjazn.pminews@sharra.llondel.demon.co.uk> <19990714.210035.0AA80202.TCaa@pierrot.co.uk>

Show Full Headers

Complete Article Headers

X-Google-Language: ENGLISH,ASCII-7-bit
X-Google-Thread: 10bb05,75d5f9974c0fd91a
X-Google-Attributes: gid10bb05,public
From: tez@pierrot.co.uk (Terrance Richard Boyes)
Subject: Re: Newshistory checking
Date: 1999/07/15
Message-ID: <19990715.002916.0AA80202.Uaaa@pierrot.co.uk>
X-Deja-AN: 501393574
X-PGP-Fingerprint: 20 66 0F E8 85 64 9C 2C  6F 82 6F 19 6C 2F 6A 1D
X-Cancel-Lock: TEZT:7c07fb37cbbdcd73b745b96a5ffe1ad1 b97cf2bceb8637d1a000d52a10a91f1a
X-PGP-Keyfile: <URL:http://www.pierrot.co.uk/pgpauth/>
References: <19990713.200500.0AA80202.PEaa@pierrot.co.uk> <slrn7oo9bq.j5g.chris@ccserver.keris.org> <abbaryybaqryqrzbapbhx.fevjazn.pminews@sharra.llondel.demon.co.uk> <19990714.210035.0AA80202.TCaa@pierrot.co.uk>
X-PGP-Auth: pub   512/3A14C4B9 1999/04/22 Tez Boyes <tez@pierrot.co.uk> iQBVAwUBN405cUXush46FMS5AQF2SQIAta3MeR06sz72mC0IaROEO9lHo816kh4A bbyA177SNsUy+z117K+M79g58V7a7Ryat/hJpD+ozESyq9ri0KzcXg== =1IkP
X-NaMLServ: $VER: v0.9b <tez@pierrot.co.uk>
X-Complaints-To: abuse@demon.net
X-Mail2News-Path: news.demon.net!finch-post-11.mail.demon.net!pierrot.demon.co.uk!pierrot.co.uk!tez@pierrot.co.uk
X-Trace: mail2news.demon.co.uk 932058126 mail2news:322 mail2news mail2news.demon.co.uk
Organization: ECL.Net
User-Agent: tin/pre-1.4-980618 (AMIGA)
X-Lan-Trace: news.pierrot.lan 19990715.002916 news:Uaaa IDENT tez@ami060.pierrot.lan
Newsgroups: demon.ip.developers

Reply to Parent Message