🚀 go-pugleaf

EGAD!  The first posting of this article got horribly mangled somehow.
As far as I can tell, groups of six lines were randomly deleted,
and the lines before and after each group spliced together.
I have no idea what happened, but since I said some harsh things I
wanted them at least to be communicated accurately, so I have canceled
the original and am re-posting.  I apologize for the waste of bandwidth.
I post dozens of articles a year; it figures that this one would be
the one to get garbled!

---- Text of original posting, as it was intended to appear: ----

[ Tirade alert!  The question does get answered, though. ]

Wow!  A question about expansion versus evaluation that I can answer!
After a mere 7 years of using TeX daily, I'm finally catching on!

Michael Barr (barr@triples.Math.McGill.CA) asks why the following
produces an error:
%%%%%%%%%%%% file starts here %%%%%%%%%%%%%%
\let\uc=\uppercase
\newif\ifucase
\def\ss{\ifucase SS\else\char"19\fi}
\def\aa{\ifucase\AA\else\accent23a\fi}
\def\ae{\ifucase\AE\else\char"1A\fi}
\def\oe{\ifucase\OE\else\char"1B\fi}
\def\o{\ifucase\O\else\char"1C\fi}
\def\uppercase#1{{\ucasetrue \uc{#1}}}

\mark{\uppercase{chapter}}
\bye
%%%%%%%%%%%% file ends here %%%%%%%%%%%%%%
Specifically, TeX complains of an "incomplete \iffalse".  Barr asks
whether this may reflect a bug in "\mark".

No, no, silly, it's *supposed* to do that, of course!  It's obvious
that TeX is doing just what you want it to do.  Right?  Well, OK, it's
not at all obvious to me, either -- I can explain it now that I've seen
it, but I never would have predicted it (a statement that is all too
true of too too much of TeX).  In any case, it is working as documented.

Here's why: macros are expanded but not evaluated inside a "\mark"
(and in some other places -- such as a \write).  So the "\uppercase"
macro inside the "\mark" gets expanded into "{\ucasetrue \uc{chapter}}",
and then "\ucasetrue" itself is expanded.  The hack that "\newif" uses
to create conditionals defines "\ucasetrue" to be "\let\ifucase=\iftrue".
If the ordinary sequence of evaluation were happening (as it is, for
example, in horizontal mode when TeX is building a paragraph) then
"\ucasetrue" would be expanded, then the tokens resulting from the
expansion would themselves be expanded (etc. etc.) until a nonexpandable
token (which \let is) was detected, and that would then be evaluated.
The evaluation of the "\let" would "gobble up" the "\ifucase=\iftrue"
tokens *without* *expanding* *them*; the assignment would be made
(locally to the enclosing group), and all would be well.  BUT inside
the "\mark" construction evaluation is suppressed, so when the expansion
of "\ucasetrue" produces the tokens "\let\ifucase=\iftrue", the "\let"
is seen to be unexpandable AND SO THE SCANNER JUST SKIPS OVER IT,
preserving it for later processing, AND ENCOUNTERS THE "\ifucase"
(which would have been "gobbled" had the "\let" been evaluated).
Now the current meaning of "\ifucase" is "\iffalse" (because when
"\newif" creates a conditional, it makes the conditional initially
false).  But "\iffalse" is handled by the expansion processor, *not*
the evaluation processor, and expansion *does* take place inside
a "\mark" (that's how we got into this mess in the first place),
so TeX goes merrily off looking for a matching "\fi" for the "\iffalse".
It doesn't find one, of course, but it does find (two tokens after
the "\iffalse") another conditional, the "\iftrue" on the other side
of the equals sign, so there are actually TWO unclosed conditionals
inside the "\mark"!  Bummer!

You see what an insane "programming language" TeX is?  Far too much
of The TeXbook is a compendium of cute hacks and absurd kludges, made
necessary by TeX's ingenious but demented evaluation strategy.

Of course, the solution is to defer the evaluation of the "\uppercase"
macro.  If we replace the next-last line of the file by

   \mark{\noexpand\uppercase{chapter}}

then the error goes away (try it).  But don't fool yourself into
thinking that the problem has been solved!  You see, we've only
deferred expansion of the single token "\uppercase"; the rest of
the mark text is still expanded.  So if the construction were
actually

   \mark{\noexpand\uppercase{\ae sop's \oe uvres}}

then there would be no error, but we'd see "aeSOP'S oeUVRES" in the
running head (or wherever it is that the mark text is used).  We would
have to type

   \mark{\noexpand\uppercase{\noexpand\ae sop's \noexpand\oe uvres}}

for our true intentions to be realized.

And that, ladies and gentlemen, is why I'll use software of Don Knuth's
to typeset my letters, or even my doctoral thesis, but I'd never ever
let him write air traffic control software or even balance my checkbook.
He is very very clever (which is good), and so are his programs (which isn't).

Incidentally, I *still* may not have it right; if not, I'd be
grateful for corrections.

Anybody rememeber Strachey's GPM?  The General Purpose Macrogenerator
was a macro-expansion tool Strachey developed to assist in bootstrapping
a CPL compiler on the Titan (Atlas 2) computer.  The GPM was a complete
(in the sense of "as powerful as any Turing machine") programming system
that worked exclusively by macro expansion.  In the article in which he
described the GPM (in the Computer Journal, sometime in 1965 -- the date
isn't on the photocopy I'm looking at) Strachey showed how to define
(very slow) routines to do integer addition and multiplication,
conditionals and loops, all with macros.  Strachey also wrote:

   ...one of the remarkable features of the GPM is its great power
   in spite of using so little apparatus... albeit at the cost of
   a very considerable obscurity of the written programs.
   It has been our experience that the GPM, while a very powerful
   tool in the hands of a ruthless programmer, is something of a
   trap for the sophisticated one.  It contains in itself all the
   undesirable features of every possible machine code---in the
   sense of inviting endless tricks and time-wasting though fascinating
   exercises in ingenuity... It can also be almost impenetrably
   opaque, and even very experienced programmers indeed tend to spend
   hours simulating its action when one of their macro definitions
   goes wrong...

Strachey could easily have been looking into a crystal ball and
pre-reviewing TeX (and Metafont, which is worse).  Poor Knuth: I think
he meant to be one of Strachey's "ruthless" types and ended up being
one of the "sophisticated" ones instead.  And poor users of TeX:
we're all in the same boat with him.  Strachey's GPM article appeared
twelve years before Knuth began work on TeX;  I never cease to wish
that he had read it and heeded its warning, but I suspect that he did
read it and was seduced by it.  Macro processing is like the Dark Side
of the Force: profoundly powerful and shockingly dangerous.

By the way, Strachey's article had a bug in it: one of the macro
examples is missing one level of quotes around one comma.  The effects
are detectable only in very special circumstances (I don't recall the
details, but it's something like "when a two-digit number is being
incremented and the second digit is a 9, calling for a carry to
be performed").  Maybe this error was only typographical, introduced
in the typesetting of the journal, but the CPL program he published
that implements the GPM also has bugs.  I find this somehow strangely
comforting:  it's not just me; *nobody* can get these damn macro systems
to work right!

--Cameron Smith
  cameron@symcom.math.uiuc.edu