🚀 go-pugleaf

> There is a bug in sed:  when processing files which do not end with a newline
> character, the last line is discarded.

Many UNIX text-file utilities will discard a (necessarily final)
text line that does not end in a newline.  Quite simply, such a
file is not a proper UNIX text file.

In article <2235@brl-tgr.ARPA> gwyn@brl-tgr.ARPA (Doug Gwyn <gwyn>) writes:
>Many UNIX text-file utilities will discard a (necessarily final)
>text line that does not end in a newline.  Quite simply, such a
>file is not a proper UNIX text file.

Who says?  Where's the definition of a 'proper' UNIX text file?

Maybe the "many UNIX text-file utilities" could do with fixing: discarding
lines that don't end in a newline seems bogus to me.

						Kay.
-- 
"A boy does not put his hand into his pocket until every other means of
gaining his end has failed."		_Tommy_, by J. M. Barrie.
			
			... mcvax!ukc!warwick!flame!kay

In article <2235@brl-tgr.ARPA> gwyn@brl-tgr.ARPA (Doug Gwyn <gwyn>) writes:
>Many UNIX text-file utilities will discard a (necessarily final)
>text line that does not end in a newline.  Quite simply, such a
>file is not a proper UNIX text file.

I think that the \User Guide to the UNIX System/ rebutts this as well as
I could:

"In the UNIX system, files have no internal structure; they are simply
a finite sequence of arbitrary characters."

A Unix file is a series of bytes, nothing more is needed to make it a
'proper' UNIX text file.  The reason that sed and some (few) others discard
the last few bytes after the last newline is because these utilities work
on 'lines' of input -- and the definition used by all (most?) of them for
a line is zero or more non-newline characters followed by a newline.
-- 

The MAD Programmer -- 919-228-3313 (Cornet 291)
alias: Curtis Jackson	...![ ihnp4 ulysses cbosgd mgnetp ]!burl!rcj
			...![ ihnp4 cbosgd akgua masscomp ]!clyde!rcj

> >Many UNIX text-file utilities will discard a (necessarily final)
> >text line that does not end in a newline.  Quite simply, such a
> >file is not a proper UNIX text file.
> 
> Who says?  Where's the definition of a 'proper' UNIX text file?

The problem is, there are several interpretations of such a file,
depending on the utility involved.  Perhaps there should be a
well-defined standard interpretation, but there isn't currently.

"A file of text consists simply of a string of characters, with
lines demarcated by the newline character."  -- from "The UNIX
Time-Sharing System" by Ritchie & Thompson

"text file, ASCII file -- a file, the bytes of which are understood
to be in ASCII code"  -- from "Glossary" in "UNIX Time-Sharing
System Programmer's Manual", 8th Ed.

"A text stream is an ordered sequence of bytes composed into lines,
each line consisting of zero or more characters plus a terminating
new-line character.  ...  The sequentially last character read in
from a text stream will, however, always be sequentially the last
character that was earlier written out to the text stream, if that
character was a new-line."  -- from ANSI X3J11/85-045

My personal choice would be similar to Ritchie & Thompson, where
newlines delimit (NOT "terminate") text lines, so that the last
character in a text file would not need to be a newline.  However,
this raises the question of what utilities should do with the
null line at the end of every text file that DOES end with a
newline; this will still be utility-dependent (and should be
documented whenever it is handled differently from other text
lines in the file).

X3J11/85-045 botched it anyhow, since they intended that ALL UNIX
files qualify as "text streams" under stdio (vs. "binary streams",
which have to be handled differently on some non-UNIX OSes).

So, how do we establish a standard interpretation for non-newline-
terminated UNIX text files?

(Discussion should move to net.unix.)

> In article <2235@brl-tgr.ARPA> gwyn@brl-tgr.ARPA (Doug Gwyn <gwyn>) writes:
> >Many UNIX text-file utilities will discard a (necessarily final)
> >text line that does not end in a newline.  Quite simply, such a
> >file is not a proper UNIX text file.
> 
> I think that the \User Guide to the UNIX System/ rebutts this as well as
> I could:
> 
> "In the UNIX system, files have no internal structure; they are simply
> a finite sequence of arbitrary characters."
> 
> A Unix file is a series of bytes, nothing more is needed to make it a
> 'proper' UNIX text file.

Tell that to "ed".  This is the second response I have gotten
this morning that shows lack of care in reading.  The magic
word "text" in front of "file" is what is called an adjective.
It qualifies (i.e., modifies) the general meaning of the noun
to produce a more specific meaning.  There are UNIX files and
there is a subset of these called UNIX text files.  Many UNIX
utilities are designed to work only with text files, which DO
have internal structure (line-oriented).  These and only these
are the files under discussion.

P.S.  I didn't realize Thomas & Yates was considered authoritative.

> > In article <2235@brl-tgr.ARPA> gwyn@brl-tgr.ARPA (Doug Gwyn <gwyn>) writes:
> > >Many UNIX text-file utilities will discard a (necessarily final)
> > >text line that does not end in a newline.  Quite simply, such a
> > >file is not a proper UNIX text file.
> > 
> > "In the UNIX system, files have no internal structure; they are simply
> > a finite sequence of arbitrary characters."
> > 
> > A Unix file is a series of bytes, nothing more is needed to make it a
> > 'proper' UNIX text file.
> 
> Tell that to "ed".  This is the second response I have gotten
> this morning that shows lack of care in reading.  The magic
> word "text" in front of "file" is what is called an adjective.
> It qualifies (i.e., modifies) the general meaning of the noun
> to produce a more specific meaning.  There are UNIX files and
> there is a subset of these called UNIX text files.  Many UNIX
> utilities are designed to work only with text files, which DO
> have internal structure (line-oriented).  These and only these
> are the files under discussion.

That still isn't entierly the point, Unix (the operating system)
doesn't know the difference between a "text" file and anything
else, the only thing it knows about are files and directories,
it does not concern itself with what is within the file.

The aplication program, such as "ed" chooses to determine that a
file that does not contain printable characters is not a "text"
file, this is purly abstract.

And just becuase the last character in the file is not a newline,
does mean that all of the characters preceding it all the way to
the previous newline are not text. Utilities such as "fgets"
determine a line when data is delimitered by a newline, if it
gets an EOF before a newline it flushes that data. Note that
fgets(3) is a library function and not a system call, thusly
it is a utility and not part of the operating system.
-- 
			Mikel Manitius @ AT&T-IS Altamonte Springs, FL
			...{ihnp4|akguc|attmail|indra!koura}!codas!mikel