🚀 go-pugleaf

Here are the consolidated answers to the original question from
Stanislav Fritz <stanf@microsoft.com>:

     There seems to be some debate on the proper sorting order of both the
     Hebrew and Arabic alphabets.  Many feel the existing Unicode table is
     inaccurate and incomplete.

     Is there any Arabic and Hebrew Linguists out there? I need the proper
     sort order and rules for both languages.  This needs to include
     diacritics, etc.

     Is there any contextual rules I need to be aware of? (eg the letter has
     more or less weight than normal depending if it is by itself or
     something like that)


Jeff
INSOFT-L List Manager

*========================================================================*
 Jeff Bowyer                         EMail:
 Computing Center		     jbowyer@cis.vutbr.cz
 Technical University of Brno
 Udolni 19, 602 00 BRNO
 Czech Republic
*========================================================================*


Sender: Amir Raveh (amirr@milcse.rtsg.mot.com)

As an Israeli (& a programmer), I can answer for the hebrew part of your
question.  Can you amplify your question:
1. What do you mean by 'rules' ?
2. Do you mean how to define the collating sequence for hebrew text strings?


Sender: Barb Eberlin (eberlin@ct.picker.com)

I'm new to the group and don't know what the Unicode table is.

However, we do have several people here who are very knowledgable in
Hebrew.

If you could mail me a copy of the table, I'll try to help.


Sender: Gene Dorr (edorr@legent.com)
        Legent Corporation

Sorting Arabic data is not as straightforward as sorting English
data.  The IBM manual "National Language Support Reference Manual"
(IBM order no. SE09-8002) has a chapter on sorting, and has
specific tables for Arabic and Hebrew (as well as many other
languages). According to the section on Arabic data, six keys are
needed to produce correct sorting of Arabic data (correct sorting
of Latin data requires four keys).

Hebrew appears to be much simpler to sort, simpler than Latin even,
since Hebrew doesn't have upper- and lower-case forms. The collation
order given in the IBM NLS Reference Manual is the same as the
sequence in which the Hebrew characters are encoded in Unicode.

More information on culturally-correct sorting is provided in another
IBM manual, "Keys to Sort and Search for Culturally Correct Results",
IBM order no. GG24-3516. I don't recall reading anything on non-Latin
scripts in this publication, but it gives a pretty clear explaination
of why sorting Latin data isn't as simple as you might think.

I would be happy to fax you the relevent pages from the IBM manual if
you like.


Sender: Glenn Adams <glenn@metis.com>
        Technical Director, Unicode Consortium

Unicode explicitly does not specify sorting order for any written language,
including Hebrew and Arabic.  You must not infer any type of collating
order from the order of characters as they appear in the standard.  A higher
level mechanism must support both the specification of the language of a
particular character and the ordering rules which apply to that language.

The order of Hebrew and Arabic characters as contained in Unicode derive
from ISO 8859-8 and ISO 8859-6 upon which these code blocks were based.
Although it is true that these two character sets did attempt to define
a rational order that could be made use of in collation, this does not
extend to Unicode because of the fact that Unicode is intended to operate
across multiple languages sharing the same script, but potentially employing
different collation orders.

As a consequence of the above, it is not possible for you to state that "the
existing Unicode table is inaccurate and incomplete" since you are applying
a measure that was explicitly excluded from Unicode's design.


Sender: Stanislav Fritz <stanf@microsoft.com>
Subject: Retraction on Arabic/Hebrew sort orders.

Please accept appologies for a bad email don't send any more responses,
and the following clarification:

I understand (better than ever) that Unicode does not imply any sort orders.

Currently, I have a sort table that I will use.  There does not seem to
be a consistant standard.

The debate concerning unicode was that the characters used versus other
"standards" seem to be inconsistant for Arabic.  However,  Unicode is
not my concern at this moment and please all Unicode people--thanks for
the response and you can stop sending them.