🚀 go-pugleaf

RetroBBS NetNews Server

Inspired by RockSolid Light RIP Retro Guy

Article View: comp.std.unix
Article #205

Embedded data in shell scripts?

#205
From: "David J. Fiander"
Date: Fri, 14 Aug 1992 20:22
85 lines
3121 bytes
Submitted-by: davidf@mks.com ("David J. Fiander")

All references are to POSIX 1003.2 D11.2, unless stated
otherwise.

The standard says (lines 11645--11651)

	When the shell is using standard input and it invokes a
	command that also uses standard input, the shell shall
	ensure that the standard input file pointer points
	directly after the command it has read when the command
	begins execution.  It shall not read ahead in such a
	manner that any characters intended to be read by the
	invoked command are consumed by the shell (whether
	interpreted by the shell or not) or that characters
	that are not read by the invoked command are not seen
	by the shell.


This seems to be attempting to ensure that a script like this

	foo=`sed 1q`
	hi there
	echo $foo
	echo Done

When fed to the shell on standard input will generate the output

	hi there
	Done

But the standard also says that (lines 5467--5471)

	When a standard utility reads a seekable input file and
	terminates without an error before it reaches
	end-of-file, the utility shall ensure that the file
	offset in the open file description is properly
	positioned just past the last byte processed by the
	utility.  For files that are not seekable, the state of
	the file offset in the open file description for that
	file is unspecified.

This means that the above script is _only_ guaranteed to work
in the case of

	sh <script

since sed is required to ensure that the file offset of the
seekable file script is properly positioned when it exits.

	cat script | sh

does _not_ have to work, according to the standard.  Yes, the
standard ensures that sed will read the right data, but the
shell may miss a large chunk out of the middle of the script
because sed is permitted to perform a buffered read, and not
reset the file offset pointer.

In fact, the only case in which both 'sh < script' and
'cat script | sh' will give the same answer on all systems is
that case in which the embedded data is all at the end of the
file, and that _all_ of that data will be consumed by the last
command executed by the shell.  That is:

	.
	.
	.
	filter -args
	data
	data

Where filter will use all of the data.

What is most unfortunate is that in order to support this one
case, there will have to be a lot of code in the shell which,
when the script is be read from standard input and standard
input is not a seekable file, reads the script unbuffered, a
single byte at a time, in order to ensure that the last command
in the file get the data it is expecting.

Is it really worth it?



Volume-Number: Volume 28, Number 93
Technical Details
Message-ID: <16h4n3INNk80@ftp.UU.NET>
Path: arc1-ca.pugleaf.net!sparky!uunet!uunet!not-for-mail

Show Full Headers
Complete Article Headers
Path: arc1-ca.pugleaf.net!sparky!uunet!uunet!not-for-mail
From: davidf@mks.com ("David J. Fiander")
Newsgroups: comp.std.unix
Subject: Embedded data in shell scripts?
Date: 14 Aug 1992 13:22:59 -0700
Organization: Mortice Kern Systems Inc., Waterloo, Ontario, CANADA
Lines: 85
Sender: sef@ftp.UU.NET
Approved: sef@ftp.uucp (Moderator, Sean Eric Fagan)
Message-ID: <16h4n3INNk80@ftp.UU.NET>
NNTP-Posting-Host: ftp.uu.net
X-Submissions: std-unix@uunet.uu.net