🚀 go-pugleaf

RetroBBS NetNews Server

Inspired by RockSolid Light RIP Retro Guy

Article View: borland.public.cppbuilder.internet.web
Article #877

Re. Parsing

#877
From: Elmar Baumann
Date: Fri, 27 Jan 2006 19:39
142 lines
4184 bytes
>Jesper wrote in Article <43da67c1@newsgroups.borland.com>
>with Subject: Parsing, on 27.01.2006 19:37:50
>
>
>Hi
>
>I have a question. Is it possible to "intercept" the information sent back 
>from a server?
>
>I mean, when I search for torrents at MiniNova (http://www.mininova.com) the 
>information that gets sent back to the webpage is organized to a webpage. Is 
>it possible to get the "raw data" and create a webpage with the links 
>yourself. I think it's called Parsing.

Hi,
i have wrote an irc-bot function that can be interesting for you,
here is the snippet:


void __fastcall TForm1::Torrentz(AnsiString Content) //Content == Searchstring
{
 int pos,begin,end;
 AnsiString TorrentSearch,line,buff,Str,Kategorie,Name,Link,url,txt,link,html;

 torrentz_work = true; // global boolean 

 TMetaClass *cl;
 TIdURI *uri;

 
 TorrentSearch = Content.SubString(10,Content.Length()-9); // get the searchstring of "!torrentz xxxxx"

 txt = TorrentSearch.Trim();

 do
 {
  pos = txt.Pos(" ");

  if(pos > 0)
  {
   txt.Delete(pos,1);
   txt.Insert("+",pos);
  }
 }
 while(txt.Pos(" ") > 0);


 url = uri->URLEncode(cl,"http://www.mininova.org/search/?search="+txt);

 try
 {
  if(txt.Length() > 0)
  {

   html = IdHTTP1->Get(url);

   if(html.Pos("No torrents found") == 0)
   {
    begin = html.Pos("Leechers");
    buff =  html.SubString(begin,html.Length()-begin);

    if(buff.Pos("cat/") > 0)
    {
     begin = buff.Pos("cat/");
     end = buff.Pos("</a>");
     Str = buff.SubString(begin+7,end-4);
    }

    Kategorie = Str.Trim();

    Str = buff.SubString(begin,250);
    line = Str;

    begin = Str.Pos("/tor");
    buff =  Str.SubString(begin,100);

    end = buff.Pos(">");
    buff = Str.SubString(begin,end-2);

    Link = "http://www.mininova.org"+buff;

    begin = line.Pos(buff);
    buff = line.SubString(begin,100);

    begin = buff.Pos(">");
    end = buff.Pos("</");

    line = buff.SubString(begin+1,end-2);

    begin = line.Pos("</a>");
    line = line.SubString(1,begin-1);

    Name = line;

    link =  "1st Mininova Result for "+txt+": �["+Kategorie+"]� "+Name+" - "+Kategorie+" - "+Link;


    if(link.Length() > 0)
    {
     TorrentResult = Link.Trim();

    }
    else
    {
     TorrentResult = "Mininova Torrent SearchEngine - No Torrents found for "+txt;
    }
   }
   else
   {
    TorrentResult = "Mininova SearchEngine - No Torrents for "+txt;
   }
  }
 }
 catch (Exception &exception)
 {
  TorrentResult = exception.Message;
 }

 torrentz_work = false;
}


The String Link contents the Hyperlink for the searched Torrent when found. In a separate
IdHTTP-Get(Link); - you could get the html. 

(Note - html-parsing does not work for all times, if mininova change it's html-outputs, you
have to adjust the code)



Regards

Elmar Baumann



------------------------------------------------------------------------------
Indy9 NNTP Internet News Reader/Writer, E. Baumann Software 2005

Portions of this Software are Copyright (c) 1993 - 2005, 
Chad Z. Hower (Kudzu) and the Indy Pit Crew 
<http://www.IndyProject.org/>
Technical Details
Message-ID: <43da7656$1@newsgroups.borland.com>
Path: arc1-ca.pugleaf.net!g2news1.google.com!news2.google.com!news4.google.com!news.glorb.com!multikabel.net!feed20.multikabel.net!newsfeed.freenet.de!border2.nntp.ams.giganews.com!nntp.giganews.com!pe2.news.blueyonder.co.uk!blueyonder!news-peer-test!btnet!newsfeed.bt.es!caladan.landsraad.net!newsgroups.borland.com!not-for-mail
References: <43da67c1@newsgroups.borland.com>

Show Full Headers
Complete Article Headers
X-Google-Thread: feeec,3e99a40fd4dd46e3
X-Google-Attributes: gidfeeec,public
X-Google-Language: ENGLISH,ASCII
From: "Elmar Baumann" <elmar-nospam@anywhere.net>
Subject: Re. Parsing
Newsgroups: borland.public.cppbuilder.internet.web
Content-Type: text/plain;
 charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit
Organization: Baumann Software NNTP
References: <43da67c1@newsgroups.borland.com>
Date: Fri, 27 Jan 2006 20:39:56 +0100
X-Newsreader: BaumannSoftware NNTP
NNTP-Posting-Host: 84.60.1.140
Message-ID: <43da7656$1@newsgroups.borland.com>
X-Trace: newsgroups.borland.com 1138390614 84.60.1.140 (27 Jan 2006 11:36:54 -0700)
Lines: 143
Path: arc1-ca.pugleaf.net!g2news1.google.com!news2.google.com!news4.google.com!news.glorb.com!multikabel.net!feed20.multikabel.net!newsfeed.freenet.de!border2.nntp.ams.giganews.com!nntp.giganews.com!pe2.news.blueyonder.co.uk!blueyonder!news-peer-test!btnet!newsfeed.bt.es!caladan.landsraad.net!newsgroups.borland.com!not-for-mail