🚀 go-pugleaf

>>>>> "tonij67" == tonij67  <tonij67@hotmail.com> writes:

    tonij67> Wierd problem going on here: I need to transfer files to a
    tonij67> OS/390 machine from a Solaris machine.  I use scp for a whole
    tonij67> lot of transfers every day with no problems so it seems like
    tonij67> the obvious choice. (I am using OpenSSH 4.3p2) We have been
    tonij67> having problems in that every time we send a ZIP file to this
    tonij67> z/os system, they are unable to decompress it.  i.e. its
    tonij67> corrupted.

    tonij67> I can send zip files to every other machine on our network,
    tonij67> and even a few machines on *their* network, all different
    tonij67> platforms.  The only one that has issues is this one OS/390.
    tonij67> I suggested they investigate the verison of SSH they are
    tonij67> running and they told me that scp expects all files to be
    tonij67> text, so the only way for this to work is for us to use sftp
    tonij67> instead of scp.

    tonij67> So I am just scratching my head here...all of the reading I
    tonij67> have done has lead me to believe that scp doesnt even *HAVE*
    tonij67> a text/binary mode, that its always binary.

    tonij67> Anyone have any idea what gives here?

Scp does not behave differently depending on the contents of a file: it
simply transmits the bytes in the same order over an SSH channel.  Since
both the file and the channel are byte-oriented, this is a well-defined
operation.  However: what the server-mode scp on the remote side is going
to do is a different story.  Copying files between different kinds of
filesystems is often *not* a well-defined operation.  If you're copying to
an existing file on a filesystem with forks, do you overwrite one fork
(and which one if there are already more than one?), or do you dump the
data into a new fork?  If you're copying to VMS, which of the 4 or 5 file
types supported by RMS (record management services) do you create?  Etc.
Scp has no way way of communicating or handling these kinds of differnces.
If the OS/390 filesystem has types that don't map well (or obviously) onto
UFS, this may be the problem.  It will come down to what
open()/create()/... in libc on the 390 actually do.

sftp may work better depending on the implementation, because the SFTP
protocol does have a notion of text files as a type and conversion to
handle different host representations.

-- 
  Richard Silverman res@qoxp.net

I don't have a precise technical answer for you but here are some
observations that I can offer:

1) If the OS/390 SSH is "OpenSSH", it should be the IBM implementation
that they fully support and not the "hacked research version" before it,
which croaked on null (0x00) bytes

2) OS/390 (and z/OS, its successor) use EBCDIC not ASCII, so there is
often a character set translation problem; from experience, I have found
that files compressed in one character set don't always decompress
(well) under another; I have to ungzip gzip'ed files in z/OS USS (Unix
System Services) before I can download them with SCP or the target file
winds up a soup of binary goulash

Workaround: uncompress the file(s) before SCP'ing them to OS/390.


On Fri, 24 Feb 2006 15:02:20 -0800, tonij67 wrote:
[snip]
> been having problems in that every time we send a ZIP file to this z/os
> system, they are unable to decompress it.  i.e. its corrupted.
[snip]
> verison of SSH they are running and they told me that scp expects all
> files to be text, so the only way for this to work is for us to use
> sftp instead of scp.
[snip]

tonij67@hotmail.com wrote:
> Wierd problem going on here:
> 
> I need to transfer files to a OS/390 machine from a Solaris machine.  I
> use scp for a whole lot of transfers every day with no problems so it
> seems like the obvious choice. (I am using OpenSSH  4.3p2)   We have
> been having problems in that every time we send a ZIP file to this z/os
> system, they are unable to decompress it.  i.e. its corrupted.
> 
> I can send zip files to every other machine on our network, and even a
> few machines on *their* network, all different platforms.  The only one
> that has issues is this one OS/390.  I suggested they investigate the
> verison of SSH they are running and they told me that scp expects all
> files to be text, so the only way for this to work is for us to use
> sftp instead of scp.
> 
> So I am just scratching my head here...all of the reading I have done
> has lead me to believe that scp doesnt even *HAVE* a text/binary mode,
> that its always binary.
> 
> Anyone have any idea what gives here?
> 
The IBM Ported Tools User's Guide states that scp assumes files are 
text. Files copied between EBCDIC and ASCII platforms are converted.

On the other hand, sftp assumes they are binary. If it is invoked on the 
z/OS side, there is an ASCII subcommand (just like for an ftp client) to 
tell it to do text conversion. Most platforms don't have that subcommand.

User1001 wrote:
> I don't have a precise technical answer for you but here are some
> observations that I can offer:

>
> 1) If the OS/390 SSH is "OpenSSH", it should be the IBM implementation
> that they fully support and not the "hacked research version" before it,
> which croaked on null (0x00) bytes

Is there a way I can verify this?  I am not too familiar with z/OS but
some familiar Unix commands seem to work:

$ uname -a
OS/390 DDC1 14.00 03 9672

$ ssh -V
OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090702f

(Right off the bat I can see they are behind on SSH, I am using
OpenSSH_4.2p1 but I dont know if the same holds true for z/OS)

>
> 2) OS/390 (and z/OS, its successor) use EBCDIC not ASCII, so there is
> often a character set translation problem; from experience, I have found
> that files compressed in one character set don't always decompress
> (well) under another; I have to ungzip gzip'ed files in z/OS USS (Unix
> System Services) before I can download them with SCP or the target file
> winds up a soup of binary goulash
>

This looks interesting.  I have no control over what the z/OS system is
doing as far as unzipping.   What is this USS you mention?  I am told
they use something called InfoZip to uncompress our files (which are
created using Solaris zip.)


> Workaround: uncompress the file(s) before SCP'ing them to OS/390.
>
>
> On Fri, 24 Feb 2006 15:02:20 -0800, tonij67 wrote:
> [snip]
> > been having problems in that every time we send a ZIP file to this z/os
> > system, they are unable to decompress it.  i.e. its corrupted.
> [snip]
> > verison of SSH they are running and they told me that scp expects all
> > files to be text, so the only way for this to work is for us to use
> > sftp instead of scp.
> [snip]

Understood, the Ported Tools Users Guide is the document that was sent
to me in defense of this issue.  What is confusing is, every other
implementation of scp is the opposite; scp uses binary. In fact, in
Solaris anyway, we dont even HAVE an option to specify text or binary.
It's always binary.

I wonder if there is any explanation as to why IBM choose to do the
opposite of everyone else?

The current IBM supported version of OpenSSH (in z/OS-1.4 USS [Unix
System Service; "Posix" {aka "unix"} in z/OS]) is,

	OpenSSH_3.8.1p1, OpenSSL 0.9.7d 17 Mar 2004


On Mon, 27 Feb 2006 07:31:40 -0800, tonij67 wrote: 
> $ ssh -V
> OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090702f