🚀 go-pugleaf

RetroBBS NetNews Server

Inspired by RockSolid Light RIP Retro Guy

Article View: comp.unix.programmer
Article #3879

Re: EUC<->Unicode (Curses woes on Solaris)

#3879
From: Paul Gorodyansky
Date: Thu, 20 Dec 2001 00:52
85 lines
4353 bytes
Hello!

Howard Kapustein wrote:
> 
> We're using curses on Solaris 8.
> Which appears to support EUC only.
> 
> int euc = wgetwch() gets a character of input and addwch(c) displays a
> character.
> 
> I need the data as Unicode.
> Specifically, UCS-2.
> [UCS-4 also works.]
> I'd even settle for UTF-8 (I've got UTF conversion routines).
> 
> How do I reliably transform EUC<->Unicode?
> 
> Locale settings don't help.
> Looks like Sun (others?) encode EUC from the keyboard as a 4-byte
> value where c & 0x30000000 == code set (see csetno() definition in
> Sun's header).
> 
> Only problem is, they don't say what code set 0, 1, 2 or 3 really is.
> Looks like CS1 == JIS-X-0208
> Yes?
> What about the rest?
> 
> And that's fine for input.
> For output, I've got a Unicode character like 0x30A1 (lower case
> katakana a) -- how do I make that a valid EUC character curses will
> recognize?
> 
> Help!
> 
>     - Howard

We ran into similar problem last year - our Application Server was
written in C (used EUC for Japanese on Solaris) and we wanted to pass
the data as Unicode to Java servlet (to our JSP layer).
It did not work and then we found why -
on a Web page (cannot find URL now) of Sun documentation where they said
that their C run-time library's
  wchar
is NOT Unicode, it's Sun's proprietary implementation.

That is, Java expected one value while our C code produced wchar
with abosolutely different value. We always use Japanese word 'test'
(te-su-to) because we wrote down the values of these 3 Japanese letters
and thus can check it during debugging:
EUC          A4C6      A4B9     A4C8
SJIS         82C4      82B7     82C6
UTF-8        E381A6    E38199   E381A8
UCS-2        3066      3059     3068        
(big-endian, it's what Java expects)

So in wchar* variable we expected to see the above UCS-2 values while
really we saw something completely different...

The solution was to have just EUC in our C code, don't do any conversion
in C/C++ code and let _Java_ do it (this works 100% OK):
 - we send EUC to Java (as a byte array), not Unicode
 - while creating Java String object from this byte array, we provide
   the encoding parameter - "EUC" - letting Java know what the source
   data encoding is.

If you don't like the above solution, then you can use a heavier one -
attach ICU library to your C/C++ code and let ICU do the conversion
from EUC to UCS-2.

Links:
1) Java's own encoding names to be used in Java conversions
  (for versions 1.2, 1.3 and 1.4):
http://java.sun.com/products/jdk/1.2/docs/guide/internat/encoding.doc.html
http://java.sun.com/j2se/1.3/docs/guide/intl/encoding.doc.html
http://java.sun.com/j2se/1.4/docs/guide/intl/encoding.doc.html

2) ICU library for C/C++:
http://oss.software.ibm.com/developerworks/opensource/icu/project/index.html


-- 
Regards,
Paul Gorodyansky
"Cyrillic (Russian): instructions for Windows and Internet": 
http://ourworld.compuserve.com/homepages/PaulGor/
Technical Details
Message-ID: <3C213653.A869531E@compuserve.com>
Path: arc1-ca.pugleaf.net!archiver1.google.com!news1.google.com!sn-xit-02!supernews.com!news.tele.dk!small.news.tele.dk!130.133.1.3!fu-berlin.de!uni-berlin.de!65.209.212.250!not-for-mail
References: <59bfe011.0112181155.77a73800@posting.google.com>

Show Full Headers
Complete Article Headers
X-Google-Language: ENGLISH,ASCII-7-bit
X-Google-Thread: f8c9b,4ecdeed075c19330
X-Google-Attributes: gidf8c9b,public
X-Google-Thread: fc89c,4ecdeed075c19330
X-Google-Attributes: gidfc89c,public
X-Google-Thread: f6653,4ecdeed075c19330
X-Google-Attributes: gidf6653,public
X-Google-Thread: 10204d,4ecdeed075c19330
X-Google-Attributes: gid10204d,public
X-Google-Thread: fb9a0,4ecdeed075c19330
X-Google-Attributes: gidfb9a0,public
X-Google-ArrivalTime: 2001-12-19 16:50:52 PST
Path: arc1-ca.pugleaf.net!archiver1.google.com!news1.google.com!sn-xit-02!supernews.com!news.tele.dk!small.news.tele.dk!130.133.1.3!fu-berlin.de!uni-berlin.de!65.209.212.250!not-for-mail
From: Paul Gorodyansky <paulgor@compuserve.com>
Newsgroups: comp.unix.programmer,comp.sys.sun.misc,comp.unix.questions,japan.comp.sun,comp.software.international
Subject: Re: EUC<->Unicode (Curses woes on Solaris)
Date: Wed, 19 Dec 2001 16:52:35 -0800
Organization: Private person
Lines: 85
Message-ID: <3C213653.A869531E@compuserve.com>
References: <59bfe011.0112181155.77a73800@posting.google.com>
NNTP-Posting-Host: 65.209.212.250
Mime-Version: 1.0
Content-Type: text/plain; charset=koi8-r
Content-Transfer-Encoding: 7bit
X-Trace: fu-berlin.de 1008809450 18222113 65.209.212.250 (16 [25072])
X-Mailer: Mozilla 4.79 [en] (WinNT; U)
X-Accept-Language: en,ru