🚀 go-pugleaf

In article <6498.269a4527@vax1.tcd.ie>, cpmurphy@vax1.tcd.ie writes:
>
>So I had a good look at the new 'ls' program that was
>in comp.sources.amiga recently. There I was alarmed to see that there is a bug
>in 1.3 to do with file blocks. Apparently using Examine() or ExNext() with a
>FileInfoBlock structure will not always give the correct number of blocks taken
>up by a file.

Well ... the real "bug" is that the only place it's "officially documented"
(that I could find) is in an old AmigaMail from CBM (the Nov/Dec 1988 issue,
in an article by Steve Beats on the differences between the FFS and the OFS).
To quote from the "struct FileHeaderBlock" listed on page 5 of that issue:

   ULONG HighSeq;         /* total blocks used in file (not updated
                             by FFS, only the old filing system) */
   ULONG DataSize;        /* number of data blocks used for this file */  (sic)

One other mention of this is made by Betty Clay in her fine article on the
FFS in an issue of the now (sadly) defunct TrasnAmi journal, from about the
same time.  In it she mentiond that a USENET posting describes this, but I was
unable to locate any such from my "archives" of "probably useful information".

As may be, once you figure out that what we're really talking about is a part
of what is known as the "struct FileInfoBlock" as defined in the dos.h/dos.i
include files, and you equate "HighSeq" with "fib_NumBlocks", and you decide
to ignore the comment associated with "DataSize" above as a typo, things
start to make sense ... :-)

Oh yes, one other thing, the comment on "HighSeq" (fib_NumBlocks) is wrong.
The field *is* updated by the FFS ... it's just not accurate (usually off by
2 or 3 blocks, but that depends on the file's size).

For example, a file I have is 135046 bytes long.  On an FFS device, the
fib_NumBlocks entry says 267 data blocks, whereas the file actually has only
264 data blocks.  And no, fib_NumBlocks isn't adding in the file's header
and extension blocks for you.  The file actually occupies 268 blocks total
(264 data, 1 header, 3 extensions).

BTW, on an OFS device (floppys, for example), this file will have 277 data
blocks, and occupy 281 blocks total.


>The way to get around it, so it says, is to use Info().

You use the call to Info() to find out the number of data-bytes per data-block
(id_BytesPerBlock in the InfoData struct).  You use this to "correct" the
value found in fib_NumBlocks (see below).


>  I'm a little bit confused by all of this. Could someone please enlighten me.

I can't imagine why ... :-)  Actually, this whole area is one of the most
poorly "documented" areas in the entire AmigaOS (along with the console/CLI
interface).  Due, I'm sure, to the braindead TriPOS/BCPL crap that had to
be grafted on top of an otherwise pretty nice OS at the last moment, many
years ago.  CBM has been working hard to exorcize this stuff, and I really
hope to see the interfaces and documentation improve with 2.0.  From the few
"bits'n'pieces" that have been mentioned here at times, I don't think I'll be
disappointed.

One other problem is that none of this information is in the RKM's.  They just
point you to the Bantam AmigaDOS manual, which tells you very little, and what
it does say is quite out of date.


>How do I get the "real" number of blocks taken up by a file so as to avoid this
>bug? Say I have a file on a HD and I want to copy it to a floppy. How do I
>ensure that there is enough space on the floppy for this file? Is it to convert
>it's size in bytes to "floppy blocks", then compare it to the Info() from the
>floppy?

Well, assuming your HD partition is an FFS one, you could just say:

    "ls -lB488 filename" 

using ls v4.0k, take a look at the number of blocks reported, and see if your
floppy has that many blocks left on it (with the AmigaOS command "Info").  See
the ls docs for further examples.

Or you could use Mike Meyer's "willfit" pgm.  I think he's calculating things
correctly, as we hashed this over in email awhile ago.

Don't use the recently posted "du" (v1.2), as it does not always give the
correct number of blocks on large files.  I've reported this to the author.

If it's for your own code, you can use/modify the routines that handle this
in ls ... fixNumBlocks() and blkalloc().  You first call fixNumBlocks(), then
blkalloc() for each file of interest (though if the file of interest is on
an OFS device, it isn't strictly necessary to do the fixNumBlocks() call).

I've appended these two little routines below (edited somewhat to remove some
"extraneous" comments :-) ).


In subsequent postings, Mike Meyer said:
>
>There isn't a bug in 1.3 about the number of blocks a file uses.
>There's just a difference of opinion between OFS and FFS about how
>what that phrase means. One of them counts the number of data blocks,
>the other counts all the blocks used by the file, both data and
>overhead. (Question to CATS: Is this considered a bug? And is it fixed
>in 2.0?)

Not quite, Mike (see the example above).  Basically, you cannot depend on
the value in fib_NumBlocks to tell you anything meaningful if the device
is an FFS one.

And no, it's not a "bug".  Since it's "documented", that makes it a "feature",
right?  :-)

What *is* a bug, is that the official 1.3 includes don't tell you that
fib_NumBlocks is meaningless on FFS (nor do the AutoDocs, RKM's, etc).


>I haven't checked 2.0 yet, but on 1.3 there's no way (that I
>know of) to determine how many data blocks/header block. On the other
>hand, everything uses 72.

There are a couple of "interesting" items that could be of further use in
Steve Beat's AmigaMail article:

"Please note that these structures assume a fixed, 512 byte disk block.  This
 is a valid assumption for filing systems up to V1.3.  However this will NOT
 be the case under V1.4 Kickstart which will support full variable sized disk
 blocks.
"

and:

"Hash table size will always be set to 72 for instance, this can be used as
 one of the consistency checks before proceeding to directly read from or
 write to the disk.
"

and (in BOLD face):

"Never use this knowledge in an application program.  It is guaranteed to
 break under V1.4 Kickstart.
"

and (of possible real use):

"struct RootBlock {
    [...]
    ULONG HTSize;       /* Size of the hash table in longwords,
                           must be set to 72 */
"

Nevermind that I have yet to find a RootBlock (etc.) struct in any of the
includes or headers.  At least there is a place for the information to
reside.  Whether or not it will actually be used is another story ... though
it would seem to be a shame to keep the 72 blocks/extent limitation if the
block sizes on some future (hypothetical) filesystem were to be of 4K size,
or even 1K for that matter.

Until this stuff gets *documented*, going with "72" is about the best you
can do, I guess.


Perhaps the "Keeper of AmigaDOS" could comment (Hi, Randell) ...?

/kim



The routines fixNumBlocks() and blkalloc() from ls v4.0k:

#define  MAX_BLKS_PER_EXTENT  72

struct InfoData *CurID = 0;  /* Global InfoData for Info() */

/*
 *  fixNumBlocks() - A hack to fix the fib_NumBlocks field so it is correct.
 *
 *  It was busted with the introduction of the FFS in AmigaDOS 1.3.  Don't
 *  ask where this is documented ... as far as I know, it isn't;  not in the
 *  1.3 includes, autodocs, readme's, user's manual, RKM's, or DevCon notes.
 *  It is mentioned (sort of) in an AmigaMail, and I'm told it was mentioned
 *  once in a message on USENET.
 *
 *  As may be ... since fib_NumBlocks was in use thruout this code before
 *  this was discovered, it was easiest to just fixup the fib_NumBlocks when
 *  the file/dir gets initially Examine/ExNext'd,  with a call to Info() for
 *  *all* files (which means alot of unnecessary calls get made [for files
 *  all in the same dir, etc]).  The performance penalty ends up being about
 *  1 sec, for a tree with 537 entries (which seems acceptable to me).
 *
 *  Helluva way to run a ship ...
 *
 *  /kim   /\;;/\
 *
 */

/* CurID is allocated in _main() during initialization, etc. */

VOID fixNumBlocks(lock, fib)
  struct FileLock *lock;
  FIB *fib;
{
    LONG *nb;
    LONG bsize;
    static int errflag = 0;

    if ((LSFlagsX & NOFIXNUMBLOCKS) != 0) return;

    if (blksize == 0)
    {
      /* Try to fill InfoData, bomb if not readable for some reason (like */
      if (Info((BPTR)lock, CurID) == 0)   /* ls'ing a "pathass'd" assign) */
      {
	/* Print error msg only once, and only if we'll be printing block counts */
	if ((errflag == 0) && ((LSFlags & (LONGLIST | TOTALIZE)) != 0))
	{
	  errflag++;
	  asprintf(workstr, "\nls: warning (%ld): block count(s) may be inaccurate - see ls.doc\n\n", IoErr());
	  WSTR(workstr);
	}
	return;
      }
      else
	bsize = CurID->id_BytesPerBlock;
    }
    else
     bsize = blksize;  /* non-0 blksize used to force the size of your */
                       /* choice ... it is set from the cmd args parse */
                       /* routine for the -B option                    */

     nb = (LONG *)&(fib->fib_NumBlocks);
    *nb = (fib->fib_Size / bsize) + ((fib->fib_Size % bsize) ? 1 : 0);
}


/*
 *  blkalloc() - Returns the actual number of blocks allocated by a file/dir
 *		 (or just the data blocks, if DATABLKSONLY is set).
 *
 *  Assumes 1.3 original or fast filesystem (and a fixed up fib_NumBlocks).
 *
 */

LONG blkalloc(fib)
  register FIB *fib;
{
    if ((LSFlagsX & DATABLKSONLY) == 0) {
      return(fib->fib_NumBlocks +
	    (fib->fib_NumBlocks / MAX_BLKS_PER_EXTENT) +
	   ((fib->fib_NumBlocks % MAX_BLKS_PER_EXTENT) ? 1 : 0) +
	   ((fib->fib_Size == 0) ? 1 : 0)  /* kludge for 0-len files (and dirs) */
	    );
    } else {
      return(fib->fib_NumBlocks);
    }
}

-- 
UUCP:  kim@uts.amdahl.com   -OR-   ked01@juts.ccc.amdahl.com
  or:  {sun,decwrl,hplabs,pyramid,uunet,oliveb,ames}!amdahl!kim
DDD:   408-746-8462
USPS:  Amdahl Corp.  M/S 249,  1250 E. Arques Av,  Sunnyvale, CA 94086
BIX:   kdevaughn     GEnie:   K.DEVAUGHN     CIS:   76535,25