🚀 go-pugleaf

In <1993Jan6.221136.27780@twg.com> "David Herron" <david@twg.com> writes:

>Taking the `forms' example.  A tag might be `<NAME>' and the value at
>that tag is the persons name.  Or a purchase requisition form might
>have <AMOUNT-REQUIRED> <ITEM-DESCRIPTION> <CATALOG-NUMBER> <UNIT-PRICE>
>and <PRICE> tags, with associated values.  Is there a concept wherein
>some `tags' have `values'?  Or does a tag only create a context, and the
>data within that context is interpreted in a particular way?

The SGML markup only defines a structural context, e.g., Ammount-Required
must precede Item-Description, or Name contains <lastname> followed
by <firstname>.  It is up to a particular processing application to
define what those elements mean, which they do by associating some
processing with the elements and their data.  This processing can be
anything you can program.  SGML neither defines nor constrains the
purpose to which you put SGML-encoded data.

Elements (tags are the syntactic constructs that delimit element
boundaries) can have values in a sense.
All elements (except one special case we can ignore for simplicity)
can be plain text (PCDATA or "parsed character data"), other
elements only, or elements and PCDATA ("mixed content").  When elements
only take PCDATA as their content, the PCDATA is generally intended
as that element's "value" in the way you mean.  Note that in
SGML, elements always contain things, they do not simply represent
start and end flags, and that containment is tracked and reported
by the SGML parser, and knowledge of that containment structure can
be used by applications to make decisions based on the containment
hierarchy for any element or data.

>In any case it appears that for `querying' type applications the
>SGML parser must build up some data structure, and that the application
>program must then be able to probe the data structure at its leisure.
>For a normal text processing application (SGML -> TeX -> PS conversion)
>you can get away with creating and destroying the data structures on
>the fly.  But to be consulted at the applications leisure the data
>structures must stay around for a long time.
>
>So... Are there any SGML programming toolkits available which allow
>for this sort of thing?  Is the ARCSGML/JClark suitable in this way?
>(I'm on a small budget)

Someone has provided a Rexx interface to ARCSGML for use with
Personal Rexx under MS-DOS.  With this, you could fairly easily
write "query" applications.  The basic idea is
to first define the data structures you need to hold the data
you want, just as you would for any other data processing
application.  You then process the document, grabbing data
off of the relevant elements as they go by and putting it into your data
structures.  The rest you just let flow through and ignore.

When you're done processing the document, you do whatever needs
to be done with the data you've collected, such as create
reports, load databases, etc.  This should be fairly straight-forward
processing.

<digression>

The examples above, and most SGML applications in existence today,
process SGML documents sequentially from start to finish.  However,
SGML does not require sequential processing, and it can make just
as much sense to define applications that work with SGML documents
wholistically as a tree, rather than sequentially, at
least in the abstract.  In this processing model, access is
via queries, rather than sequential access by waiting for the
element you want to flow by.  Some SGML-based online presentation
systems, such as EBT's Dynatext, present this sort of view of
documents, providing query-based direct access to elements and
and data, which makes sense in a dynamic, direct access application
like online presentation.

You can also define processing models that use both sequential
and query-based processing.  For example, you might use a query
to access the root of some subtree within a document, and then
process that subtree sequentially to build some output, or in
the course of sequential processing, you might use a query to
do lookahead to resolve a cross reference (as opposed to
doing some sort of two-pass process).  This sort of system might
make the most sense when your documents are already stored in
an SGML-knowledgeable database system that provides the retrieval
function you need.

The point is that even though bytehead hackers like myself who
knock together quick programs to do pragmatic stuff tend to
think about processing SGML sequentially, especially because
that's what our tools give us, SGML processing is not limited
to that one processing model, and that by thinking of SGML documents
as databases against which you can do queries, you might start
to think of very interesting and useful ways to use your
SGML documents you might not have thought of before.

Eliot Kimber                      Internet:  drmacro@ralvm13.vnet.ibm.com
Dept E14/B500                     IBMMAIL:   USIB2DK9@IBMMAIL
Network Programs Information Development     Phone: 1-919-543-7091
IBM Corporation
Research Triangle Park, NC 27709

[Eliot Kimber]
:
|   <digression>
|
|   The examples above, and most SGML applications in existence today,
|   process SGML documents sequentially from start to finish.  However,
|   SGML does not require sequential processing, and it can make just as
|   much sense to define applications that work with SGML documents
|   wholistically as a tree, rather than sequentially, at least in the
|   abstract.  In this processing model, access is via queries, rather than
|   sequential access by waiting for the element you want to flow by. ...

This is a good point, but an important distinction needs to be made (and
"at least in the abstract" needs to be stressed).

The SGML document consists of one or more _entities_, each a character
sequence, which represents a fragmented linearization of the element
structure.  This fragmented linearization needs to be parsed (and
reconstructed) before we can access the element structure.  The element
structure can fruitfully be regarded as the _product_ of the parser (cf.
ESIS), and the parser has to parse the document "from start to finish" to
be able to build this structure.

According to the standard, the only way you can possibly parse an SGML
document is sequentially (cf. 6.2 SGML Entities), and any access other than
sequential in a data stream is to entities in it (cf. SDIF).

This problem has been discussed before, in relation to what #CURRENT meant
as an attribute default value declaration.

In order to access elements in the element structure non-sequentially, you
have to build some sort of index into the entity structure, or store the
data returned from the sequential parsing process.  Since element contents
can span entities (one of SGML's significant strengths), the demand on
software that accesses the unparsed entity structure is high, compared to
storing a fully parsed SGML document in some form optimized for retrieval
by the application.  This task becomes daunting in the presence of frequent
short references, character entity references, and references to other
internal entities.

Several interesting problems follow from the constraints that this
represents.  (No, that's not the "understatement of the year" -- it's only
January 7th. ;-)

Best regards,
</Erik>
--
Erik Naggum                 ISO  8879 SGML                    +47 295 0313
                            ISO 10744 HyTime
<erik@naggum.no>            ISO  9899 C                 Memento, terrigena
<SGML@ifi.uio.no>           ISO 10646 UCS             Memento, vita brevis

In <19930107.006@erik.naggum.no> Erik Naggum <SGML@ifi.uio.no> writes:
>[Eliot Kimber]
>:
>|   <digression>
>|
>|   The examples above, and most SGML applications in existence today,
>|   process SGML documents sequentially from start to finish.  However,
>|   SGML does not require sequential processing, and it can make just as
>|   much sense to define applications that work with SGML documents
>|   wholistically as a tree, rather than sequentially, at least in the
>|   abstract.  In this processing model, access is via queries, rather than
>|   sequential access by waiting for the element you want to flow by. ...
>
>This is a good point, but an important distinction needs to be made (and
>"at least in the abstract" needs to be stressed).
>
>The SGML document consists of one or more _entities_, each a character
>sequence, which represents a fragmented linearization of the element
>structure.  This fragmented linearization needs to be parsed (and
>reconstructed) before we can access the element structure.  The element
>structure can fruitfully be regarded as the _product_ of the parser (cf.
>ESIS), and the parser has to parse the document "from start to finish" to
>be able to build this structure.

Erik's completely correct.  In my mind, I've already abstracted away
the entity structure, such that the only SGML-defined constructs I'm
thinking about are enlement structures.  You can do this by defining
semantics that provide "entity-like" functions for elements.  For
example, if you use the HyTime content extraction link idea to access
the content of SGML elements, you don't have to depend on entities
to do data organization.  All data organization can be done using
only SGML elements.  This allows you to view the SGML data without
regard to how it might be organized into system entities, or rather,
you view the data as a single, giant entity that is an entire "document"
or set of documents.  In addition, if you constrain all elements and
semantics to the element hierarchy, you remove all requirements to
process the document sequentially (ignoring the fact that the SGML
must be *parsed* sequentially, which is not the same thing).  This
means that valid subtrees can be processed in isolation because
they are, by this definition, completely self contained.  Or, thought
of another way, there are no longer any processing states that
change asynchronously from the element hierarchy itself.  This is very
powerful for things like re-use and modularization.

New documents are built using content
references to re-use objects rather than using entity references
(except for data entities, of course).  You can thus think of
the database as a "containment-only document" that serves only as
a repository of elements for use elsewhere, at its simplest,
a one-element document type with a content-model of ANY.  If you
define the first level of containment as defining the object
boundaries, hey presto, a database of objects.  If you make this
structure recursive (which is to SGML by dint of using ANY),
you can have arbitrarily complex sets of containers within containers.
This enables the data organization to be completely different from
the presentation or retrieval organization, letting you
modularlize your data to any granularity desired (down to the
character level, unless you use HyTime addressing to address
bits within characters--just joking).  At all levels of granularity,
the same basic SGML features or application semantics are used
to manage structure and addressing (e.g., containment hierarchy
and ID or query-based hyperlinks).

In other words, you can define all data organization of SGML
data purely in terms of element structure without ever using
external entities (not counting data entities, of course).  HyTime
defines the processing semantics you need in terms of hyperlinks,
which provides a nice formalism, but you don't need HyTime to
define those semantics for a given application.

One of the big revelations for me as I've worked at designing
SGML applications is that SGML puts very few limits on what
you *can* do.  I found that I tended to limit my own thinking by
what I was used to doing, and when I stopped doing that, I
realized I had a lot of freedom to define whatever processing
and application semantics I wanted to.  This is because SGML does
not define the processing semantics in any way.  Charles Goldfarb
has always been very adamant on this point and I finally see
why.  Of course, there's no reason you can't define another
standard set of semantics in SGML terms, but the possible number of
such definitions is infinite.  I also found that I could define,
for my application, which happens to be a text processing
application, all my semantics in terms of hyperlinks, which means
they can all be expressed in terms of HyTime, for the
most part.  I now have an application whose data structures
and semantics are expressed almost entirely in terms of existing
international standards, which provides, at the very minimum,
a coherent, "universal", language for expressing my application
design, and that is completely divorced from implementation specifics.
I find this to be pretty darn cool and very liberating.

Eliot Kimber                      Internet:  drmacro@ralvm13.vnet.ibm.com
Dept E14/B500                     IBMMAIL:   USIB2DK9@IBMMAIL
Network Programs Information Development     Phone: 1-919-543-7091
IBM Corporation
Research Triangle Park, NC 27709