🚀 go-pugleaf

    find takes about 2 secs to traverse the same tree, which is still
    pretty slow, but faster than TeX.

Is GNU find faster than TeX?  If so, something strange is going on,
because ...

    What I did find was that the code is incredibly bright about working
    out if a directory *has* subdirs -- it counts the links and assumes
    that if there are only 2 then it hasn't.

... I stole this trick (fortunately not patented yet, at least to my
knowledge) from GNU find.  A user (I forgot who, unfortunately) posted
the fix to bug-gnu-utils.

I suggest running tex under gdb (or some debugger), and seeing if in
fact the ``incredibly bright'' code is actually working.  As you
describe your setup, the only directories that should be readdir'd are
the top-level ones, which is only 4 or 5.  Obviously it is stat-ing all
the hundreds of input files.  The question is why.  If you have
any subdirectories at all, the link-counting trick fails, and all
directory entries must be stat-d.

You also suggest one-level subdirectory searching:

    If there is in fact no way to get it usable, then a compromise scheme
    might be to have single-level subdir searching.  

This is what I implemented first (before I came across the link-counting
trick); it got released in 5.84, I think.  I don't see why this would be
any faster than the recursive case, though, if the directory hierarchy
is right for subdir searching.  (If it isn't, you'll probably lose
either way.)

You also suggest, as a final stopgap, having TeX etc. read directory
paths from a file.  I'd prefer not to have an additional configuration
file around for installers to have to think about.

I'm not sure if I made any significant changes to the path searching
code after I released 5.851c.  But you might want to grab dvipsk.tar.Z
or xdvik.tar.Z and copy the common files into web2c's lib/ directory,
just in case.

By the way, messages like this about web2c are usually best directed at
me via mail (copy to the list as well if you want).  It was purely
fortuitous that I happened to be reading news for a little while before
your article expired.

Thanks for the note.
--
karl@cs.umb.edu
Member of the League for Programming Freedom---write to league@prep.ai.mit.edu.