🚀 go-pugleaf

In article <3C86BC4F.9930D425@vernier.com>,
 Christopher Corbell <ccorbell@vernier.com> wrote:

>It seems that newlines-to-CR does odd things,
>and even if unchecked some odd things can happen.

well... no, not really odd... but possibly not the thing you might 
expect until it's explained to you :-)

First, what does 'map newlines to cr' mean? It means that when that 
setting is on and you write '\n', the compiler generates a CR character, 
and when you write '\r', the compiler generates a LF character. 
Normally, Mac OS newlines are CR.

Also, std::endl is the same as '\n'.

Finally, when you are writing to a file which is opened in text mode 
using the standard C library, any LF characters are converted to CR, and 
vice versa, if you are using the default MSL. (See buffer_io.c)

So, on to explain what you see.

>With newlines-to-CR off, writing a std::endl
>to an fstream writes '\r' 

std::endl is '\n', '\n' is LF, LF is converted to CR when written to the 
actual file.

>but writing std::endl
>to a stringstream writes '\n'.  

std::endl is '\n', '\n' is LF.

>However writing
>a std::string which contains carriage returns to
>an fstream translates '\r' to '\n'!!!

Writing to a text file (which is what an fstream is by default) converts 
LF to CR.

>With newlines-to-CR on, I noticed a really bizarre
>thing: writing a literal '\r' to an fstream produces
>a '\n' in the file.  

That's because now your '\r' generates an LF, and LF is sent to the file

>Which isn't map-newlines-to-CR,
>it's map-CR-to-newline!
>
>Is all of this documented somewhere?  I searched
>the manuals for "newline" and nothing came up.
>
>What I'd really like is to have std::endl always
>be the same thing regardless of whether it's
>going to an fstream or a stringstream, and
>also have '\r' in code always mean '\r' in
>the byte and its subsequent i/o, and sim. for
>'\n' to always mean '\n', with no translation
>other than what the programmer does explicitly.
>If this is possible in MSL C++, what does 
>it involve other than the newlines-to-CR setting?

Why do you want that? The underlying principle here is simple: the 
representation of a newline in memory is a LF character, and on disk is 
a CR character. If you use text mode to open files (which fstream does 
by default), the conversion is done on the way from memory to disk and 
back.

However, if you want to do it, all you have to do is use an MSL compiled 
with no CR/LF conversion, which would be a MSL C PPC (NL), and you can 
compile it from the MSL project. But, I really think that if you are 
trying to do that, you are probably doing something wrong, which is why 
I would like to know why you think it's crucial to you to make the two 
match.

hth

meeroh
-- 
Hire me: <http://meeroh.org/hire.html>

In article <3C86BC4F.9930D425@vernier.com>,
 Christopher Corbell <ccorbell@vernier.com> wrote:

>What I'd really like is to have std::endl always
>be the same thing regardless of whether it's
>going to an fstream or a stringstream, and
>also have '\r' in code always mean '\r' in
>the byte and its subsequent i/o, and sim. for
>'\n' to always mean '\n', with no translation
>other than what the programmer does explicitly.
>If this is possible in MSL C++, what does 
>it involve other than the newlines-to-CR setting?

Just to add to what Meeroh explained 

If you do not want end of line translations to occur you must open a 
file in binary format.    (Apple uses one char to signify and end of the 
line, while UNIX uses another, and Widnows yet another).

You should not change the newline to CR in fact we are dropping this in 
future IDE's so it can only be changed by a pragma  to avoid this 
confuseion.  It is there to use with MPW files and normally should not 
be changed or altered.

I hope this helps expand on the excellent answer Meeroh gave.

Ron

-- 
       TOP PICK   HandHeld Computing - issue 4.5
   "The best overall development tool for the Palm OS"

 Metrowerks, maker of CodeWarrior  -  "Software Starts Here"  
Ron Liechty - MWRon@metrowerks.com - http://www.metrowerks.com

meeroh wrote:

> >What I'd really like is to have std::endl always
> >be the same thing regardless of whether it's
> >going to an fstream or a stringstream, and
> >also have '\r' in code always mean '\r' in
> >the byte and its subsequent i/o, and sim. for
> >'\n' to always mean '\n', with no translation
> >other than what the programmer does explicitly.
> >If this is possible in MSL C++, what does
> >it involve other than the newlines-to-CR setting?
> 
> Why do you want that? The underlying principle here is simple: the
> representation of a newline in memory is a LF character, and on disk is
> a CR character. If you use text mode to open files (which fstream does
> by default), the conversion is done on the way from memory to disk and
> back.

The reason I want that is also simple, I think: I develop 
a cross-platform application, and use standard C++.
Our file io code is shared across platforms.
I think that's what the standard library is for.

We have one text-based file format, and files 
saved from one platform must open fine on the other 
platform.  The newline issue as whitespace in the XML
file is a convenience: we want users to be able to 
look at the file in a native text editor.  Internally,
however, some elements contain text in which 
whitespace is significant, including newlines; here 
(precisely because the file could have come from another 
platform)  we need to test and translate newlines to 
see if they match the current platform's native 
linebreak before handing the text off to a native
text-display object or edit field.

It isn't enough to say, "I'm a Mac app and I want
\n to be \r, always".  The general solution would be,
"I'm a platform with newline sequence x+ and if
I encounter newline sequence y+ I want to translate
it to x+", and have this to hold true for both
input and output to any iostream.  But to implement
this in cross-platform code, you need consistency
from the development environment.

VC++6 maps std::endl to "\r\n" when writing to
a file, but if you substitute a literal "\r\n" you get
exactly what you asked for - you don't get "\r\r\n".
Needless to say it never gives you "\n\r".

If MSL and VC++ both fudged the standard in the
same way we could still develop a general solution,
but in this one case MSL is harder to deal with.
(This -is- the exception - in almost every other
std:: issue we've run into MSL has been far more
sane and compliant than the Redmond camp).

> But, I really think that if you are
> trying to do that, you are probably doing something wrong, which is why
> I would like to know why you think it's crucial to you to make the two
> match.

Thanks for the information on what it's really doing.
However on this issue, I do believe that CodeWarrior 
has it wrong; there should always be a way to turn
off literal substitutions and have an ANSI character
escape sequence mean what it means.

Also, I don't quite see why you don't recognize
the inconsistency in the terminology.  If the 
functionality is "Reverse CR and LF", that is 
what it should be called - "Map newlines to CR" 
indicates a one-way translation.  It is also 
inconsistent in that when the setting is -not- 
checked, LF's output to an fstream still get 
mapped to CR, and CR's inside a std::string get
mapped to LF when the std::string is output via <<.

Of course I can still make it work and have managed to
fix the crucial problems, but it involves more complication 
and inconsistency, e.g. using a different newline for output
to an fstream than to a stringstream, which is unfortunate -
being able to treat iostream subclasses equivalently whenever
possible is a big deal in projects like this one.
And being able to count on CR==CR and LF==LF would have
just made it all a lot easier.

Thanks for the discussion.

Chris

MW Ron wrote:
[....]
> (Apple uses one char to signify and end of the
> line, while UNIX uses another, and Widnows yet another).
[....]

This is true, and that's why I need literal
CR's and LF's to remain what-they-are - that file
I'm opening could have come from my program running
under Windows or Linux.

Thanks,
Chris

In article <3C87D3BD.8F0436CD@vernier.com>, Christopher Corbell
<ccorbell@vernier.com> wrote:

| MW Ron wrote:
| [....]
| > (Apple uses one char to signify and end of the
| > line, while UNIX uses another, and Widnows yet another).
| [....]
| 
| This is true, and that's why I need literal
| CR's and LF's to remain what-they-are - that file
| I'm opening could have come from my program running
| under Windows or Linux.

There is a prepackaged solution for what you want.  It is spelled:

std::fstream s("data.dat", std::ios::binary);
                           ^^^^^^^^^^^^^^^^

Binary mode does nothing but suppress newline translation during file
I/O.

-- 
Howard Hinnant
Metrowerks

In article <3C87D1AC.1521FAB0@vernier.com>,
 Christopher Corbell <ccorbell@vernier.com> wrote:

>The reason I want that is also simple, I think: I develop 
>a cross-platform application, and use standard C++.
>Our file io code is shared across platforms.
>I think that's what the standard library is for.

Then you shouldn't use std::endl, either, because there are no 
guarantees what character (or characters!) it will produce.

>We have one text-based file format, and files 
>saved from one platform must open fine on the other 
>platform.  The newline issue as whitespace in the XML
>file is a convenience: we want users to be able to 
>look at the file in a native text editor.

If you want users to be able to look at the file in a native text 
editor, then you must use native newlines. That means that your files 
need to be converted when going from platform to platform. But you say 
that you don't want that?

>Internally,
>however, some elements contain text in which 
>whitespace is significant, including newlines; 

Encode the newlines when you write them to the file. That way you will 
know what the file contains, always, and it will not be translated, 
ever. Oh, but then your users need to know to do that when they edit the 
file in an editor.

>Thanks for the information on what it's really doing.
>However on this issue, I do believe that CodeWarrior 
>has it wrong; there should always be a way to turn
>off literal substitutions and have an ANSI character
>escape sequence mean what it means.

There is. Don't turn on CR/LF remapping, and don't open files in text 
mode (see what Howard said).

>Also, I don't quite see why you don't recognize
>the inconsistency in the terminology.  If the 
>functionality is "Reverse CR and LF", that is 
>what it should be called - "Map newlines to CR" 
>indicates a one-way translation.  It is also 
>inconsistent in that when the setting is -not- 
>checked, LF's output to an fstream still get 
>mapped to CR, and CR's inside a std::string get
>mapped to LF when the std::string is output via <<.

Yes, the name is awkward. 

>Of course I can still make it work and have managed to
>fix the crucial problems, but it involves more complication 
>and inconsistency, e.g. using a different newline for output
>to an fstream than to a stringstream, which is unfortunate -
>being able to treat iostream subclasses equivalently whenever
>possible is a big deal in projects like this one.
>And being able to count on CR==CR and LF==LF would have
>just made it all a lot easier.

Like we said, open the stream in binary mode.

hth

meeroh
-- 
Hire me: <http://meeroh.org/hire.html>

In article <3C87D3BD.8F0436CD@vernier.com>,
 Christopher Corbell <ccorbell@vernier.com> wrote:

>MW Ron wrote:
>[....]
>> (Apple uses one char to signify and end of the
>> line, while UNIX uses another, and Widnows yet another).
>[....]
>
>This is true, and that's why I need literal
>CR's and LF's to remain what-they-are - that file
>I'm opening could have come from my program running
>under Windows or Linux.

Well then you need to open a file in Binary mode so there is no text 
translation.  That is what it is for.

I don't understand why you think this has anything to do with Map 
Newlines.  That setting is only there for usinglibraries compatible with 
MPW.   If you are using this now for anything other than this you are 
misusing it and are sure to have problems.

Ron

-- 
       TOP PICK   HandHeld Computing - issue 4.5
   "The best overall development tool for the Palm OS"

 Metrowerks, maker of CodeWarrior  -  "Software Starts Here"  
Ron Liechty - MWRon@metrowerks.com - http://www.metrowerks.com

In article <3C87D1AC.1521FAB0@vernier.com>,
 Christopher Corbell <ccorbell@vernier.com> wrote:

>If MSL and VC++ both fudged the standard in the
>same way we could still develop a general solution,

The solution is there it is written in the standard and you are not 
using it.  We will not fudge our tools to be less compliant.

>but in this one case MSL is harder to deal with.
>(This -is- the exception - in almost every other
>std:: issue we've run into MSL has been far more
>sane and compliant than the Redmond camp).

If you write your code to be standard compliant then it will work on 
both  CodeWarrior and that other brand or GCC or Borland or whatever. :)

I don't like piling on but I think you want are confusing the end of a 
line marker with  the escape sequences \n and \r or the 0xA, 0xD values.

Ron

-- 
       TOP PICK   HandHeld Computing - issue 4.5
   "The best overall development tool for the Palm OS"

 Metrowerks, maker of CodeWarrior  -  "Software Starts Here"  
Ron Liechty - MWRon@metrowerks.com - http://www.metrowerks.com