Article View: comp.std.internat
Article #676Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST)
From: Boyd Roberts
Date: Fri, 08 Jan 1993 09:27
Date: Fri, 08 Jan 1993 09:27
45 lines
2267 bytes
2267 bytes
In article <1993Jan7.033153.12133@fcom.cc.utah.edu>, terry@cs.weber.edu (A Wizard of Earth C) writes:
>
> Consider a newline terminated text database containing fixed length lines,
> or consider a database consisting of variant text records in fixed fields.
> In either case, the amount of data per field is now variant on Runic encoding.
> For instance, if we accept the Plan-9 soloution, an application used in
> both England and the US will vary as to how much data is representable per
> fixed field based on whether or not that data contains the English "#"
> character. This gets worse the further you get from base ASCII coding.
Using Plan 9 utf you know the maximum size in bytes that a Rune can
be encoded into. Fixed fields only have to be a multiple of this value,
defined to be to be UTFmax. So where's the problem?
We can take this piece of code as working example (ie. functioning)
which takes an array of Runes, encodes them and then calls write
to send them in a reasonably efficient manner (ie 1 Rune != 1 write):
void
sendrunes(Rune *r, ulong len)
{
char s[128 * UTFmax];
while (len)
{
char *p;
for (p = s; p <= &s[sizeof s - UTFmax] && len; len--)
{
int n;
n = runetochar(p, r++);
p += n;
}
(void)write(fd, s, p - s);
}
}
Runes are your friend.
Boyd Roberts boyd@prl.dec.com
``When the going gets wierd, the weird turn pro...''
Technical Details
Message-ID:
<1993Jan8.092754.6344@prl.dec.com>Path:
arc1-ca.pugleaf.net!sparky!uunet!haven.umd.edu!decuac!pa.dec.com!decprl!decprl!boydReferences:
<2615@titccy.cc.titech.ac.jp> <1993Jan5.090747.29232@fcom.cc.utah.edu> <id.EAHW.92A@ferranti.com> <1993Jan7.033153.12133@fcom.cc.utah.edu>Show Full Headers
Complete Article Headers
Newsgroups: comp.std.internat Path: arc1-ca.pugleaf.net!sparky!uunet!haven.umd.edu!decuac!pa.dec.com!decprl!decprl!boyd From: boyd@prl.dec.com (Boyd Roberts) Subject: Re: Dumb Americans (was INTERNATIONALIZATION: JAPAN, FAR EAST) Message-ID: <1993Jan8.092754.6344@prl.dec.com> Keywords: Han Kanji Katakana Hirugana ISO10646 Unicode Codepages Sender: news@prl.dec.com (USENET News System) Nntp-Posting-Host: spooky.prl.dec.com Organization: Digital Equipment Corporation - Paris Research Laboratory References: <2615@titccy.cc.titech.ac.jp> <1993Jan5.090747.29232@fcom.cc.utah.edu> <id.EAHW.92A@ferranti.com> <1993Jan7.033153.12133@fcom.cc.utah.edu> Date: Fri, 8 Jan 1993 09:27:54 GMT Lines: 45