Data directories for Grid computing: SRS-LDAP test

Article View: bionet.software.srs

Article #132

Back to bionet.software.srs Previous Next

#132

From: Don Gilbert
Date: Thu, 11 Jul 2002 10:28

42 lines
3663 bytes

Dear folks w/ interests in bio-data distribution,

I've put together an experimental SRS6-LDAP gateway tool,
available at 
http://iubio.bio.indiana.edu/grid/directories/

This is in a very preliminary stage, essentially it is just
far enough along to assess its efficiency for bulk bio-sequence
search and retreival via LDAP (lightweight directory access protocol).  

If you are interested in trying it out, there are two simple
ldapsearch programs (java and perl) here, which have test
cases to the IUBio Archive SRS system.  They should work on most
computers.  A basic LDAP url for this is
  ldap://iubio.bio.indiana.edu:3895/srv=srs
which serves the same databanks as
  http://iubio.bio.indiana.edu/srs/

The source code for the SRS6-LDAP backend is included here, but I
haven't gotten documentation to a usable point - anyone trying
this out, let me know where I can help (besides the SRS6 binary
distribution you will need openldap2 (www.openldap.org) and a
C-Perl tool (for ldap to srs query conversion).

In comparison to the SRS6 tools "getz" (commandline) and "wgetz"
(web backend), it looks efficient and capable.  E.g. it performs
about the same speed over network as the local getz program, and
about 3x faster than wgetz over http, without adding any binary
encoded sequence formats (something you can do in LDAP, which uses
binary encoded ASN.1 as its transport stream).

E.g., it takes just a second or two to do a query, but to retrieve
sequence the time is proportional to number of sequence
records: 1.4 minutes for 20,000 records;  7 minutes for 350,000
records,  30 minutes for some 1.2 million records in GenBank
which mention 'human'.

-- Don Gilbert
-- d.gilbert--bioinformatics--indiana-u--bloomington-in-47405
-- gilbertd@bio.indiana.edu
---

Technical Details

Message-ID: <200207091553.g69FrWK14435@cricket.bio.indiana.edu>

Path:

arc1-ca.pugleaf.net!archiver1.google.com!news1.google.com!newsfeed.stanford.edu!bloom-beacon.mit.edu!hgmp.mrc.ac.uk!net.bio.net

Show Full Headers

Complete Article Headers

X-Google-Language: ENGLISH,ASCII-7-bit
X-Google-Thread: fa6ec,6b092aae241489ea,start
X-Google-Attributes: gidfa6ec,public
X-Google-ArrivalTime: 2002-07-11 03:28:27 PST
Path: arc1-ca.pugleaf.net!archiver1.google.com!news1.google.com!newsfeed.stanford.edu!bloom-beacon.mit.edu!hgmp.mrc.ac.uk!net.bio.net
From: gilbertd@bio.indiana.edu (Don Gilbert)
Newsgroups: bionet.software.srs
Subject: Data directories for Grid computing: SRS-LDAP test
Date: 11 Jul 2002 11:28:19 +0100
Organization: BIOSCI/MRC Human Genome Mapping Project Resource Centre
Lines: 43
Sender: mod-srs@hgmp.mrc.ac.uk
Approved: BIOSCI Approval Key <biosci-approval-key@net.bio.net>
Message-ID: <200207091553.g69FrWK14435@cricket.bio.indiana.edu>
NNTP-Posting-Host: mercury.hgmp.mrc.ac.uk
X-Trace: niobium.hgmp.mrc.ac.uk 1026383300 13559 193.62.192.80 (11 Jul 2002 10:28:20 GMT)
X-Complaints-To: news@net.bio.net
NNTP-Posting-Date: Thu, 11 Jul 2002 10:28:20 +0000 (UTC)
X-Received: from localhost (localhost [127.0.0.1])
 by mercury.hgmp.mrc.ac.uk (Postfix) with ESMTP id 248F117AAD
 for <bio-srs@net.bio.net>; Tue,  9 Jul 2002 16:50:39 +0100 (BST)
X-Received: from cricket.bio.indiana.edu (cricket.bio.indiana.edu [129.79.225.189])
 by mercury.hgmp.mrc.ac.uk (Postfix) with ESMTP id 8072017B31
 for <bio-srs@net.bio.net>; Tue,  9 Jul 2002 16:50:33 +0100 (BST)
X-Received: (from gilbertd@localhost)
 by cricket.bio.indiana.edu (8.11.6+Sun/8.11.6) id g69FrWK14435
 for bio-srs@net.bio.net; Tue, 9 Jul 2002 10:53:32 -0500 (EST)
X-To: bio-srs@net.bio.net
X-Razor-id: 007340f1ba686c1371432638a3cbbf815e4eaf43
X-Auth: PGPMoose V1.1 PGP bionet.software.srs
 iQCVAwUAPS1dv6CIqsWvyF5FAQHwVwQAuoU4tNy76vN41tEgAO4G4hdUgoXKGvHB
 V40HaYmd8L4ogpUMT1VJYr0/TTBSQ9iAaaQXx3Ri0QaCJuOzuKjxUr+KXCAx9Ku2
 MInasW98Bue9Ihh1UExB8XEgN5BTrZSojWm16K2tIyqKE+90qqPbzg9k0FjxVYWO
 0Pe0bmJBrxs=
 =UAfx