🚀 go-pugleaf

In article <1993Jan23.204214.29441@netcom.com> kschwarz@netcom.com (Kenneth L. Schwarz) writes:

> Does anyone know how sorts are implemented on lists of Japanese words?
> For example, let's say we have a database with 10 fields per record, and
> these records contain Roman characters, kanji, or kana, or any combination
> thereof.  If we want to be able to sort this database with any of the fields as
> the sort key, what do we do?

> I know that in Japan telephone books are sorted by aiueo jun (that is, by the
> phonetic order of a kana matrix.)  And CDs are sorted in a similar way
> in record stores.  So it seems likely that a phonetic sort is the most 
> useful organization for human's to deal with.

> But that would not be so for computers.  Sorting kanji by stroke count is
> easy:  there is one and only one stroke count for any particular glyph.
> But to do a phonetic sort would require that the algorithm recognize the
> particular reading of the character in context.  This is a bear of a
> problem.

> Still, there must be ways that Japanese databases handle such problems.
> Anybody know how it's done?

If there are no external requirements imposed upon the definition of your
sort, why not sort by the underlying binary representation of the kanji.

(Then all you have to worry about is big-endian vs. little-endian
representation of integers.)
--
                                   Dan Prener (prener@watson.ibm.com)

In article <1993Jan23.204214.29441@netcom.com>, kschwarz@netcom.com (Kenneth L. Schwarz) writes:
> Does anyone know how sorts are implemented on lists of Japanese words?
> For example, let's say we have a database with 10 fields per record, and
> these records contain Roman characters, kanji, or kana, or any combination
> thereof.  If we want to be able to sort this database with any of
> the fields as 
> the sort key, what do we do?

Ken, and anyone else interested should pop over to comp.std.internat
where just this discussion (seems to be getting more heated everytime
I read it so maybe argument... :).
--
jamiller@kuhub.cc.ukans.edu
James Miller

In article <1993Jan23.204214.29441@netcom.com> kschwarz@netcom.com (Kenneth L. Schwarz) writes:
>Does anyone know how sorts are implemented on lists of Japanese words?

I'm afraid you have to be more specific about why or for what purpose
you need the sort.

>For example, let's say we have a database with 10 fields per record, and
>these records contain Roman characters, kanji, or kana, or any combination
	I believe you mean "each of the fields".
>thereof.  If we want to be able to sort this database with any of the fields as
>the sort key, what do we do?

As said above, this depends on why you need the sort for.
Basically, each Japanese character is encoded by two bytes.
This includes kana and kanji, as well as the two-byte version of
alphabets, numerals, and symbols in the ASCII set (and even Greek and
Kyrilic alphabets).  The one-byte (ASCII) and two-byte codes can be
mixed, although we won't go into the details here.
If all you need is some well-defined ordering, then simply sorting
the byte sequences by eg. dictionary ordering will do.

>I know that in Japan telephone books are sorted by aiueo jun (that is, by the
>phonetic order of a kana matrix.) 

Side note: to be precise, no.
Telephone books use a kind of mixed indexing scheme.
The initial characters (kanji) are sorted by aiueo ordering of their readings,
but then, those with the same kanji are clustered, regardless of
how the following characters are read.
Within the cluster, the rest of the characters are sorted by the same
scheme.  Thus, for entries involving homophones (same reading, different
kanji), the result is different from strict aiueo ordering.

>And CDs are sorted in a similar way in record stores.
>So it seems likely that a phonetic sort is the most 
>useful organization for human's to deal with.

Yes.

>But that would not be so for computers.  Sorting kanji by stroke count is
>easy:  there is one and only one stroke count for any particular glyph.

Um, but stroke counts are only good for sorting only single characters
(as in a kanji dictionary (Kanwa-jiten)).  It is useless for sorting
character strings.

>But to do a phonetic sort would require that the algorithm recognize the
>particular reading of the character in context.  This is a bear of a
>problem.

Yes it is.

>Still, there must be ways that Japanese databases handle such problems.
>Anybody know how it's done?

Surprise.
The method used in most databases (which is the most practical solution)
is to prepare multiple fields for the same entry, eg. a field with
kana/kanji mixed, field with only kana (which gives the reading),
or that in roma-ji (alphabets).
Phonetic sort can be done with the latter two.
Well, sorting in aiueo order is still not straightforward, because
simply sorting by the character code order won't do.  Imagine sorting
alphabetic sequences with upper/lower cases mixed.  The situation is
similar, though more complicated.  But it is possible to give an
algorithm to do this.

The details of these will be quite long, so won't be given here.

Hope this helps.

-Yuzuru Hiraga

Sorting of Japanese script data is not as simple as you think.  See Chapter 2
of my book The Fifth Generation Fallacy (Oxford, 1987).  (A Japanese version
is now available:  Konpyuuta syakai to kanzi, SIMUL, 1992.)
-- 
J. Marshall Unger, Chair
Hebrew & East Asian Languages & Literatures
2106 Juan Ramon Jimenez Hall 
University of Maryland