Article View: demon.ip.developers
Article #733Re: SMALLNET.EXE, HTTP command extension; & Killfile question
From: John Washington
Date: Sat, 09 Nov 1996 00:00
Date: Sat, 09 Nov 1996 00:00
121 lines
5263 bytes
5263 bytes
Cross-posted to demon.ip.support.pc since this is no
longer a blue-sky development issue, but available software!
I've only compiled the 'tiny' TextWin version so far, and
will be uploading that to
http://www.wash.demon.co.uk/tw/tinytw.exe
today. If I get positive feedback on that I'll compile the
other versions.
On Wed, 30 Oct 96 16:23:12 GMT, in article
<846692592snz@bobsh.demon.co.uk> Bob@bobsh.demon.co.uk
in demon.ip.developers, "Robert. S. Harris." wrote:
RH> I have been using the Smallnet version of the KA9Q software for a
RH> while, using the new HTTP command, I've been very pleased with it, but
RH> now have a suggestion as an extension to its functionality.
RH>
RH> Scenario:
RH> I produce a list of URLs to get, into a text file called htm.lst in
RH> the Demon rootdir.
RH> I use the command at the "net>" prompt - http @htm.lst - and all are
RH> fetched, assuming they exist and I got the URLs right.
RH> I filter them through an HTML converter, making sure I keep all the
RH> href's so I can build the next list.
RH>
RH> Problem:
RH> Most local pages have local references which is fair enough, except
RH> that I can't find a reference in the page I'm scanning, to show what
RH> the original URL was to form a complete line for the new list.
RH> i.e. Web pages seldom say where they came from.
RH>
RH> Suggestion:
RH> When I used to use the Lynx servers and used the [p]print/[m]mail-to
RH> options, the returned messages had this original URL as subject line,
RH> to ensure that the source of the page was always known.
RH> Would it be a good idea for the HTTP command to add this information
RH> into the specified or deduced output file, something in the form of a
RH> comment like -
RH> <!source: www.someware..com/path/filename.html>
Thanks for the suggestion Bob, and thanks to all those who
clarified what is/isn't/might-(not)-be legal HTML or something
that browsers will barf on.
Accordingly there's a new feature in the ka9q HTTP file grabber,
so that 'label' does pretty much what Bob suggests. So, at the
'net>' prompt
http //wash.demon.co.uk/index.htm label
fetches that page and puts a line on the front. This line will
not be visible in a browser, but if you examine the text of the
file it's the very first line.
If you use the script file facility, e.g.
http @myscript
obviously you can put 'label' on any or all of the lines
that you wish. However, it seems likely that some users
will want to 'label' all the files they fetch. It would be
nice to allow
http @myscript label
to work in the way that you'd hope. Rather than make that a
special case, I've implemented it by taking any parameters
on that line and appending them to every line in the script
file. Obviously that's not very useful for some parameters,
e.g. I don't imagine anyone will ever use 'file=xxx' in this
way, since all the fetches in the script would overwrite the
same file!
However, it may well be useful with the 'force' or (perhaps)
'link' options.
On Fri, 01 Nov 96 18:01:15 GMT, in article
<846896475snx@warren.demon.co.uk> leveret@warren.demon.co.uk
in demon.ip.developers, "Nick Leverton" wrote:
NL> Ok, here's an obscure one for y'all ...
NL>
NL> I put a line in my NNTP killfile as follows:
NL> Subject: *[ _-]LOOT[ _-]*rail*
NL> wanting to catch "lootrail", "loot-rail", "loot_rail" and "loot rail"
NL> and similar combinations (if you read misc.transport.urban-transit
NL> you'll understand).
NL>
NL> But it also catches a whole lot more. Setting nntp trace to 3 shows
NL> me that it also kills stuff like:
NL>
NL> Subject: Re: Poly marriage contract
NL> Subject: Re: More Junk Email
NL> Subject: Re: "Name of a dog! Rover
NL>
NL> Looking at the code for wildmat(), I guess it is getting confused about
NL> the hyphen and imagining it is part of a character range. A hyphen as
NL> first or last in the string ought to be treated as a literal, but
NL> wildmat doesn't conform to this convention.
I've put in the fix Graham recommended for this.
On various occasions, too numerous to list, in articles whose
message IDs are too long for my eidetic memory, people
have written things like:
> > Wouldn't it be nice to have wildcards in mail kill files...
> Yes, it's a simple matter of..
[one line fix snipped]
and Demon have mobilised huge teams of programmers to update Ka9q
accordingly... NOT! :-(
Well, that fix also is in this version. :-)
--
John Washington
Technical Details
Message-ID:
<847543433snz@wash.demon.co.uk>Path:
arc1-ca.pugleaf.net!unknown!not-for-mailReferences:
<846692592snz@bobsh.demon.co.uk>Show Full Headers
Complete Article Headers
X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 10bb05,2dcae2933831a457,start X-Google-Attributes: gid10bb05,public X-Google-Thread: fb525,2dcae2933831a457,start X-Google-Attributes: gidfb525,public From: john@wash.demon.co.uk (John Washington) Subject: Re: SMALLNET.EXE, HTTP command extension; & Killfile question Date: 1996/11/09 Message-ID: <847543433snz@wash.demon.co.uk> X-Deja-AN: 195540593 x-mail2news-path: wash.demon.co.uk references: <846692592snz@bobsh.demon.co.uk> x-mail2news-user: john@wash.demon.co.uk newsgroups: demon.ip.developers,demon.ip.support.pc