🚀 go-pugleaf

N.B. This article is rather long because the problem is rather important
     for me - the prospect that I can loose (some of) my data and/or
     programs at any time is not a very nice one - and I spent a lot of
     time trying to solve and prevent the problem.
     If you are in a hurry, then please jump to the "Questions" to see
     if you can help or contribute.

N.B. This article contains the names of several products in a somewhat
     negative context. I do not want to imply in any way that these are
     bad products. I only describe my findings using these products. The
     actual cause of my problems may be not in these products at all.

The problem
===========

  I get, more often than I consider normal, read errors on my diskettes:
At least one problem sector on each of 5 out of 20 diskettes.

Details of the problem
======================

- With "read errors", I mean the DOS error
  "Data error reading drive B
   Abort, Retry, Ignore, Fail ?"
  from commands like "copy b:file.ext c:".
  Other programs like Norton's 4.5 DT (DiskTest) and NDD (Norton Disk
  Doctor) and PC Tools' 6.0 Verify File/Disk give similar errors.

- I get the read errors only for 3.5"/1.44MB diskettes, but that is
  probably because I hardly use any other type (i.e. 5.25"/360KB/1.2MB
  or 3.5"/720KB).

- Most of the time the read errors are "hard errors", i.e. trying a
  little later and/or removing/reinserting the diskette does not help.
  Only once I had a "soft error".

- When a read error occurs on a particular diskette, it normally occurs
  only for a single sector.
  On one diskette there were, over time, two problem sectors with the
  following scenario: Read error in sector X, reformat diskette, problem
  gone, several months go by, read error in sector Y.
  On one diskette, the TDK one (see below), I got, over time, three
  problem sectors, probably because I had not reformatted the diskette
  after encountering the first problem sector.

- One problem diskette, the TDK one (see below), I tried to read on two
  other systems:
  - A system of the same brand as mine (Hewlett-Packard) and with the
    same brand and type of diskette drive.
  - A system of a friend of my son. I have no details of this system,
    but is was not a Hewlett-Packard system.
  I got exactly the same read errors, in the same sectors, on those
  other systems, so it is safe to assume that there is nothing wrong
  with my diskette-drive, at least not for reading.

- The majority of the problems, at least one problem sector on each of
  5 out of 20 diskettes, occured on JVC MF-2HD diskettes. This caused me
  not to buy this brand of diskettes anymore.
  Nearly all JVC diskettes have the same batch number (production
  identification): H971458RY. One problem diskette has a different batch
  number, which is hard to read, probably H972688RY.

- My other 3.5"/1.44MB diskettes, about 80-100, are mainly TDK MF-2HD
  diskettes. Until recently, I never had a problem with these TDK
  diskettes, but then I also got a (hard) read error on one of them
  (which was hardly ever used). This TDK diskette has batch number
  JB907B0106. After trying to recover the data from this TDK diskette, I
  got two more (hard) read errors in other sectors of this same
  diskette.

- I never had a problem with diskettes which were not written on my
  system, i.e. for example diskettes of software products which I
  bought. But of course I read these kinds of diskettes less frequently
  than my other diskettes.

- I never get an error when formatting or writing a diskette.
  When I started to get the read errors, I checked each COPY operation
  to a diskette by doing a COMP directly after the COPY. In the majority
  of the cases, the problem sectors were detected by this "COMP after
  COPY" operation.
  While the problem is not *detected/reported* during formatting or
  writing, it may well be *caused* by/during formatting or writing.

- I am under the impression that the use of the PC Tools (6.0) PCFORMAT
  program might be related to the problem.
  I use PCFORMAT all of the time, because it has more features than the
  basic DOS FORMAT.
  My son uses both PCFORMAT and DOS (in the beginning MS-DOS 3.3 and now
  DR DOS 6.0) FORMAT on my system and has never had any problems, also
  not with his 20 JVC MF-2HD diskettes. Of his JVC diskettes, which have
  different batch numbers than mine: GO32857R and GO32858R, about half
  are formatted with PCFORMAT and about half with FORMAT (Norton's (4.5)
  DI (Disk Information) gives 'PCFormat' in the 'system ID' field if the
  diskette was (last) (really) formatted with PCFORMAT and 'IBM  3.3' if
  it was formatted with FORMAT.).
  The exception to these formatting habits seems to be the one TDK
  diskette with the read errors. Its 'system ID' field says 'IBM PNCI'.
  I do not know where this ID comes from. It could be the result of a
  DISKCOPY of an unknown diskette onto this one (DISKCOPY also copies
  the bootsector, so also the 'system ID', from the source disk to the
  destination disk). However that is not very likely because there are
  no deleted files on this diskette, which there (probably) would be
  after a DISKCOPY, followed by a DEL *.*, followed by COPYing the
  files, which it currently contains, onto it.

Why am I worried?
=================

  The fact that the problem now also occurs on a non-JVC diskette
worries me, and is the main reason for me to write this article: When
will my other 80-100 diskettes start to give problems?

  I use my diskettes mainly for archiving purposes. They contain my own
text and data files, my own programs, freeware and shareware programs,
etc.. Of course I make backup of my hard-disk, but I hate to have to
make a backup copy each time (i.e. whenever the content changes) of all
my diskettes too, just because one might get bad.

Data recovery attempts
======================

  Attempts to recover the data in the sector which gives a read error,
have, in general, been unsuccessful. I have the Norton Utilities 4.5 and
PC Tools 6.0.

- Before trying any of the below data recovery methods, it is wise to
  make a backup copy of the diskette with the read error(s). This is
  wise because during the data recovery attempts, more problem sectors
  may appear. By making, as soon as possible, a backup copy of the
  still-good part of the problem diskette, data loss is minimized.

  Make sure to keep/make the problem diskette write-protected when
  trying to make a backup copy!

  However one can not make a backup copy with DISKCOPY, because DISKCOPY
  will fail with "Error reading from the disk" when it tries to read a
  problem sector and DISKCOPY will terminate (i.e. NO "Abort, Retry,
  Ignore, Fail?" message).

  If one has enough space on the hard-disk, or on another diskette in a
  two diskette-drive system, then one can make a diskimage-in-a-file
  copy of the problem diskette with Norton's NU (probably called
  DiskEdit in newer releases), and copy that diskimage-in-a-file to
  another backup diskette.

  NU has a sector mode and a file mode for reading and for writing. The
  mode, sector or file, can be set seperately for reading in the "Choose
  item" menu and for writing in the "Write item to disk" menu.

  The backup copy can be made by reading the whole diskette (sectors 0
  through 2879 for the example 3.5"/1.44MB diskettes) in sector mode and
  writing it in file mode to another disk(ette) drive (from now on this
  will be called sectors-to-file mode), and then copying the diskimage-
  in-a-file in file-to-sectors mode (starting sector 0) to the backup
  diskette.
  During the sectors-to-file copy, NU will say
  "The copy may not be correct
  [several blank lines]
   Check the destination to verify correct copying"
  because it tried to read one or more unreadable sectors.

- The most "safe" recovery method is to use NU to try to copy the
  content of a file, which contains problem sector(s), to a file on
  another disk(ette). Norton's DT can tell which file(s) contain(s) (a)
  problem sector(s) (Actually DT will give the cluster number. NDD will
  give the sector number.).

  This data recovery method does not alter the problem sector(s) in
  any way (the problem diskette can be write-protected), so you can try
  it over and over again.

  One can copy the files in either file-to-file or sectors-to-file mode
  (and in some cases also file-to-sector mode)

  If there is only one problem sector in a particular file, then file-to-
  file mode is the easiest.

  If there is more than one problem sector in a particular file, then it
  is probably best to use sector(s)-to-file mode to copy the good and
  bad parts of the source file to individual destination files, because,
  when using file-to-file mode, in one try NU might be able to read one
  problem sector correctly, but might not be able to read the other
  problem sector(s) correctly, while in a next try the situation might
  be reversed. By reading in sector mode, and reading one problem sector
  at a time, one can try over and over again until one has recovered the
  best possible data from the problem sector(s). The individual
  destination files can be recombined with (for example)
  copy /b good1+bad1+good2+bad2+good3 total

  If the original file is fragmented, i.e. does not occupy just one set
  of contiguous clusters/sectors on the disk, then it might be easier to
  first copy the file, in file-to-file mode, to an empty diskette - i.e.
  the destination file will be unfragmented - and then copy the problem
  sector(s), in sector-to-file mode, from the source sector(s) to
  file(s), and finally copy, in file-to-sector mode, the file(s) to the
  correct sector(s) in the destination file.

  In each of the above cases, NU will say
  "The copy may not be correct
  [several blank lines]
   Check the destination to verify correct copying"
  when it encountered one or more unreadable sectors. The data which NU
  read will be written to the specified destination (file or sector(s)).
  I.e. NU will write what it has read, while DISKCOPY will just
  terminate on a read error and COPY with 'Ignore' will copy nonsense
  from the problem sector to the end of the file (COPY with 'Abort'
  (with or without an earlier 'Retry') or 'Fail' will truncate the 
  destination file at or before the failure point.).

  In my data recovery attempts, the data, which NU recovered from the
  problem sector(s), was similar to a known-good copy, but there were
  always one or more bytes (upto 83) incorrect (determined with a UNIX
  "cmp -l" clone for MS-DOS). In some cases there was only one single-
  bit error in a problem sector. For most non-text files however, a
  single-bit error is as bad as a totally corrupt file.
  (Luckily, in all-but-one of the cases, I still had an extra known-good
  copy somwhere.)

- A somewhat less safe/convenient data recovery method is the "surface
  scan" facility of PC Tools and the Norton Utilities. In this case the
  utility does only a single read, or a series of reads, of the problem
  sector, and then copies what it has read to another sector and marks
  the problem sector as bad. If this data recovery attempt is
  unsuccessful, which it is most of the time, then you have to mark the
  problem sector again as "good" (which can only be done with Norton's
  DT), and then start all over again.

- The least safe/convenient data recovery methods are the "/R" option of
  PC Tools' PCFORMAT, the "Revitalize a floppy" function of PC Tools'
  DISKFIX and the "Revive a Defective Diskette" function of Norton's
  NDD. These tools do a single read, or a series of reads, of a sector
  and then reformat and rewrite the data. If the read(s) is/are
  unsuccessful, then bad luck, the incorrect data will have been
  rewritten to the diskette, and there is no way to try again to get the
  correct data.

Questions
=========

1. Do you find at least one problem sector on each of 5 out of 20
   diskettes a high/medium/low failure rate?

2. What are your experiences with read (or format or write errors) on
   diskettes?

3. Do you have bad/good experience with the PC Tools PCFORMAT program,
   compared to the DOS FORMAT program?
   If PCFORMAT is suspect, can you recommend DISKFIX' "revitalize" or
   NDD's "revive" to try to make my diskettes more reliable than they are
   now? (It is hardly practical to copy each diskette to another
   diskette or the harddisk, do a real format of the original diskette
   and copy back the data to it.)

4. Do you know a better data recovery method than the ones described
   above?

5. Can you explain why I had better results with NU than with COPY?
   (NU: single-bit to 83 bytes wrong in the copy of a problem sector.
   COPY with 'Ignore': everything in the destination file, from the copy
   of the problem sector to the end of the file, is wrong.)
   Are there any special features in the diskette drive/controller
   hardware, BIOS or DOS, which NU uses to try to recover as much data
   as possible, which COPY does not use?

6. Do you know any, preferably freeware or shareware, programs for
   - more thoroughly formatting diskettes?
   - more thoroughly testing diskettes?
   - testing the diskette drive?

7. Any other information, tips, remarks, questions, etc.?

  Thanks in advance for your help.

Frank Slootweg