🚀 go-pugleaf

Jens M Andreasen <jens-and@dsv.su.se> wrote:

>My point is that we do not have anything but alfabetsoup before 
>evaluation. You are expecting a compiler to know the meaningfulness
>of random characters without letting it actually read it ? Then by what 
>method shall we find the hidden comma expression (Or any other doubtful 
>expression, including the works of Shakespeare)? Intuitively, but in no 
>way proven, this would involve parsing of a yet unknown nature.

Perhaps a bit of background on lexing and parsing would clear up your
questions.  They're not really related to the C standard, so I'll try to
keep it short: the first step in compiling some source code ( following
preprocessing) is to break the source code up into tokens.  Basically,
the compiler is required to read in as many characters as it can, and
call that a token.  That token is then classified as something like a
number, an identifier, a punctuator, etc.  This much is in the lexer.

The next step is the parser, which takes individual tokens, and checks
whether they're legal parts of the current item being parsed.  E.g. when
it encounters 'int' by itself, it starts parsing a declaration or
definition.  It "knows" that the next item needs to come from one of a
small group of items: optional storage class, optional qualifier (
volatile or const or both ) followed by an identifier.  This must be
followed by either a semicolon or else an equal sign followed by a
constant expression.  At this point the constant expression is read in
token by token.  Again, at each step the parser has a list of exactly
what items are legal at this point in a constant expression ( It's
longer so I won't try to enumerate it here) but one thing that's NOT in
the list is a comma expression.  For instance if it reads in a number,
it will then have a list of exactly what operators can come next:
arithemetic operators, bitwise operators, and so on.  But a comma won't
be in the list, so as soon as it encounters the comma, it stops and
gives a diagnostic.  Note that as yet, NONE of the expression has been
evaluated at all - it's still only getting broken up into constituent
parts to see if what's there is legal.  Until the compiler has figured
out what the individual bits _mean_ it can't possible put them together
and evaluate the overall value.  In this case it finds that the
individual bits don't legally mean anything, even though the meaning
they "should have" may be quite obvious.

>Or to put it another way: At what time of compile are we going to tell 
>the user that his code is way off?  

>1) When we know what it means.

>2) Before we know what it means. 

As soon as the parser finds that the expression isn't legal.  From one
viewpoint, yes, this is before the compiler has decided what the
expression really means.  

[ ... ] 

>You found the comma, and you realized that it did not belong there. This 
>is fantastic (no sarkasm here, I do believe life is beutiful) but I can't 
>see any way of formalising this general ability into a machine method. 

I'm not sure how well it would be received, but perhaps a question in
comp.compilers about how state machines are used in creating parsers
would be in order.  Quite frankly, it took quite a few people quite a
while to come up with a formalized, scientific method of doing this and
I'm quite certain I could never have done anything close on my own; I
rather doubt most of the posters here could have singlehandedly either
for that matter.  OTOH, they have been thought up, and there certainly
are now ways of turning the general ability into a machine method,
including parser generators that automatically convert a specification
of a grammar similar to that in Appendix B of the standard, into an
actual parser that will more or less automatically determine whether
some specific source code actually follows the grammar or not.

Note that Appendix B of the standard would have to be modified to
actually create a correct parser - by Appendix B, the original bit of
code would be considered legal.  The gramar in the appendix doesn't
include all constraints.  This isn't a self contradiction in the
standard because the appendices are specifically excluded from the
standard.

Now having been off-topic for _far_ too long a message, I'll shut up for
a while.
    Later,
    Jerry.

/* I can barely express my own opinions; I certainly can't
 * express anybody else's.
 *
 * The universe is a figment of its own imagination.
 */