🚀 go-pugleaf

In article <1992Sep8.113126.1@cbrown.claremont.edu>, dhosek@cbrown.claremont.edu writes:
|> The problem is that text-sorting algorithims are greatly
|> complicated by having multiple characters representing a single
|> sorting unit. This is a general problem of course (for example,
|> in Spanish, ch, ll and rr are each treated as single letters so,
|> e.g., lo comes before llegar in an alphabetized list of Spanish
|> words, on the other hand, in an American English dictionary, i
|> and i-with-diaresis would be treated as the same letter for
|> sorting purposes). The approach of coding diacriticals separately
|> has both advantages and disadvantages and I don't think that
|> it's quite the closed book represented here.

The general problem of sorting multigraphs, symbol sets in which
diacritics are taken into account in special ways, etc., can be
solved with sorting handles.   A handle is essentially a function of
strings that generates a set of strings that can be easily and
properly collated.  I was introduced to this idea by Dr. Robert Hsu,
of the Department of Linguistics at the University of Hawaii, though
I don't believe he claims to have originated it.  He discusses a
variety of techniques for constructing handle functions in a book he
is writing on linguistic data processing.

This has nothing to do with Unicode, of course.  It's already needed
with good old ASCII, even for English, e.g., to handle treating upper
and lower case letters alike.  However, knowing that there are well
understood techniques for sorting complexly collated symbol sets may
be reasuring to anyone facing Unicode for the first time!

----
John E. Koontz (koontz@bldr.nist.gov)

Disclaimer:  Views and recommendations, express or implied, are my own, and
do not reflect the opinion or policy of my employers.