🚀 go-pugleaf

In article <2171@enea.UUCP> sommar@enea.UUCP(Erland Sommarskog) writes:
>
>value. But isn't a character a more complicated data type
>than just a simple enumeration type? In some languages the
>combination may constitute a new letter ("a" with ring and dots,
>"o" with dots in Swedish), in other you can apply accents and
>other signs without affecting the sorting. (E.g. French, Italian)
>  I think that the simple represenatation for charcters is completely
>due the dominating position of the English language in the computer
>world. If computers had been invented in France the problem would
>have been solved. (And if they had been Swedish, Englishmen would

It gets even more complicated:  in Spanish, I believe, ch is considered
a separate letter, between c and d in alphabetical order (likewise with ll).

It only goes to show that alphabetical order is language-dependent, and
identical strings will sort differently depending on locale.  The only
general solution is to have intelligent operating system routines to
handle sorting.

Despite 7-bit ASCII, which makes possible code such as
   if (c >= 'A' && c <= 'Z')
there is no reason why the numeric representation of a character should have
anything to do with the position of that character in a collating sequence.

-- 
UUCP:  ... !mnetor!lsuc!maccs!gordan              BITNET: GP@TANDEM
"Sumasshedshii vsekh stran, soyedinyaites'"        Gordan Palameta

>   I think that the simple represenatation for charcters is completely
> due the dominating position of the English language in the computer
> world. If computers had been invented in France the problem would
> have been solved...

Surely you jest!  If the French had invented computers, the official
rationale for FRSCII (or whatever it would be called :-)) would go to
great lengths to explain why cedillas were part of God's plan but no
civilized human being would ever put pairs of dots above letters!
The best summary I've ever heard was "those who speak English don't
really believe that other languages exist; those who speak French know
that other languages exist, but can't understand why".
-- 
Support sustained spaceflight: fight |  Henry Spencer @ U of Toronto Zoology
the soi-disant "Planetary Society"!  | {allegra,ihnp4,decvax,utai}!utzoo!henry

In article <176@hobbes.UUCP> root@hobbes.UUCP (John Plocher) writes:
>Theory:
>	  The correct way of sorting (or comparing) strings which contain
>	accented characters is to first compare the strings with all accents
>	stripped. If the two strings are the same, then and only then are the
>	accents used.  This second comparison involves only the accents.
>	You can think of this as comparing the two strings with all the letters
>	stripped.

Sorry, that is not the correct way to sort in Swedish.
The three letters a(with a ring), a(with two dots), o(with two dots)
always come at the end of the alphabet, after z.
Traditionally v and w are also considered the same letter in Swedish.

There is unfortunately no universal way to sort things alphabetically,
each language has its own ways. This fact has for instance been incorporated
into the Macintosh system, where there sorting depends on the national
version.

	Dan Sahlin              (dan@sics.uucp)

In a recent article gordan@maccs.UUCP (Gordan Palameta) writes:
>t umlaut or q cedilla would probably be used very rarely, nor is it likely
>that anyone would go to the trouble of designing a font to accomodate such
>characters.  Another cost of such generality would be that accents and other
>marks would probably have to be indicated by escape sequences in conjunction
>with the unmodified letter.  This would make string-processing software more
>complicated (and slower), and text would be longer.

If you had something like the 8th bit meaning that the following byte is a
modifier, this would quite moderately increase the length of the text and the 
string-processing time. This solution does however not solely address the
problem that different languages have different collating sequences.

>Not at all, just define a 256-byte lookup table in an include file, and modify
>the code to
>     if (coll[c] >= FIRST_CHAR && coll[c] <= LAST_CHAR)
>with very little loss of efficiency.  To accomodate perverse languages like
>Spanish and Polish which insist on two-letter combinations for sorting,

Spanish and Polish aren't more perverse than English.
  Of course I know about look-tables. I have myself written a programme
that uses a two-level look-up table for comparing words. (And the words
are transcribed in three levels. You don't want the hyphen in a hyphenated
word to be significant.) But to have that in every single programme that
does string comparisons. No, thank you. It does increase the complexity
and the readability of the code.
  It would be much more nice if "ch1 >= ch2" meant that ch1 comes before
or at the same position as ch2 in alphabet we currently have chosen. (It's 
unclair what equality is when modified letter are involved. Probably you will
need two kinds of equality.)

>Never mind the French; what if things had turned out differently in 1588
>with the Armada, and the Spanish had invented computers?  Or the Chinese?

I just took the Frenchmen as an example, OK? No matter who had invented
the computers; if their language also had had the dominating position that
English have, that language would have set the standard for character
representation with no other language in mind. I took French as example
since they have plenty of for sorting non-significant modifiers. 

>Followups to alt.universes.

If you find the subject that uninteresting, why did you ever write
the article at all?
-- 

Erland Sommarskog       
ENEA Data, Stockholm    
sommar@enea.UUCP        

In article <719@maccs.UUCP> gordan@maccs.UUCP (Gordan Palameta) writes:
>t umlaut or q cedilla would probably be used very rarely, nor is it likely
>that anyone would go to the trouble of designing a font to accomodate such
>characters.  Another cost of such generality would be that accents and other
>marks would probably have to be indicated by escape sequences in conjunction
>with the unmodified letter.  This would make string-processing software more
>complicated (and slower), and text would be longer.

The existance of something like four different 94-character "Latin" sets
suggests that 8 bits wouldn't suffice anyway, although I haven't counted the
exact number of existing glyph combinations. I believe Welsh includes some
strange accented consonants, but I don't remember which (maybe w^). If you
also take Vietnamese into account, which allows several accents used at the
same time, you'd definitely overflow the table. TeX provides for arbitrary
combinations of accents, and I think this approach is quite simple (although
I don't suggest TeX for the encoding scheme to be used for files in general).

I don't think somebody manually has to design a font for each and every
combination, as the acute accent over e looks pretty much the same as the
acute accent over o, and the combination could be done automatically at
display-time. Some characters will need special treatment though, like
capital Swedish A with circle above (they should usually touch each other)
and Polish bar-crossed L. The amount of programming and CPU power to be
used for this depends on what quality and resolution of display you require.

If this general approach turns out to be the most practical one technically,
some people may of course go hog wild putting circles under X and cedillas
over 7, but there is as little point in stopping them as in preventing
people from writing "fiYw#s" with a proportional font. Just apply the
general accent attachment rule and they'll be quiet...

>     if (coll[c] >= FIRST_CHAR && coll[c] <= LAST_CHAR)
>with very little loss of efficiency.  To accomodate perverse languages like
>Spanish and Polish which insist on two-letter combinations for sorting,

What about the thing "Mac" or "Mc" in English (Scottish?) proper names?
I agree this example is a little extreme in comparison to the Spanish
"graphemes" ch, ll and rr (?), as well as czech ch. Maybe the English
don't mind seeing "McDonald" sorted after "Machiavelli", or whatever the
rule is/was - has it been abolished by now?

There are different kinds of sorting even within one language, depending on
the context. Donald E. Knuth provides a wonderful collection of rules for
bibliographic use in the beginning of his "Fundamentals ..." volume on
Sorting & Searching, such as ignoring articles and spelling out numbers.
These rules don't apply to filenames in a UNIX directory, I think!
-- 
Anders Andersson, Dept. of Computer Systems, Uppsala University, Sweden
Phone: +46 18 183170
UUCP: andersa@kuling.UUCP (...!{seismo,mcvax}!enea!kuling!andersa)

In article <2201@enea.UUCP> sommar@enea.UUCP(Erland Sommarskog) writes:
>>     if (coll[c] >= FIRST_CHAR && coll[c] <= LAST_CHAR)

>  Of course I know about look-tables. I have myself written a programme
>that uses a two-level look-up table for comparing words. (And the words
>are transcribed in three levels. You don't want the hyphen in a hyphenated
>word to be significant.) But to have that in every single programme that
>does string comparisons. No, thank you. It does increase the complexity
>and the readability of the code.

Nobody coding an application should of course ever have to implement those
string comparison routines explicitely over and over again, but rather
refer to generec library routines, like isalpha(c). Several libraries can
be provided for different kinds of sorting (if you want "RFC666.TXT" after
"RFC-INDEX.TXT" but "RFC888.TXT" before because of the way numbers are
spelled out, that's up to your choice). If the string comparison library
is too big to be linked into your 42 executables, then provide it as a
sharable image, or (in an emergency) put it in your favourite kernel...
-- 
Anders Andersson, Dept. of Computer Systems, Uppsala University, Sweden
Phone: +46 18 183170
UUCP: andersa@kuling.UUCP (...!{seismo,mcvax}!enea!kuling!andersa)

In article <718@maccs.UUCP> gordan@maccs.UUCP (Gordan Palameta) writes:
:>In article <8410@utzoo.UUCP> henry@utzoo.UUCP (Henry Spencer) writes:
:>) [if the French had invented computers, ........
:If the French had invented computers, there is little doubt that a character
:set to support French would have appeared sooner.  .....
:There is no chance that a 16-bit character set would have sprung up, fully
:formed -- computer memory used to be very, very expensive (weren't characters
:six bits, once upon a time?)

It would be variable length, from 5 to 14 bits ... but the last three
or four aren't pronounced :-).   More seriously, while a given
numeric character representation doesn't correspond identically to the
collating sequence, (viz. English [Aa]<[Bb]... vs ascii or ebcdic),
one can build a table listing character representations in sequence,
and use it for sorting rather than building the sequence into the
language, as was suggested with #pragma language(Franglais).  While one
might use the representation directly when the collating sequence
doesn't really matter (e.g. building unique lists), one can still build
library functions to compare words.
-- 
#				Thanks;
# Bill Stewart, AT&T Bell Labs 2G218, Holmdel NJ 1-201-949-0705 ihnp4!ho95c!wcs