🚀 go-pugleaf

On Thu, 24 Sep 92 14:43 GMT  <CGP@STARLINK.LEICESTER.AC.UK>said:
>Astronomers are using FITS tables, especially binary tables, for a wide
>range of applications from star catalogues to photon-event lists.  As
>William Pence has already noted, FITS is starting to be adopted as a
>storage format in its own right and not just as a data interchange
>format.

 Although I am personally questioning the use of FITS for storage, I
 believe the points raised by Clive Page below concerning use of FITS
 files as databases AND MORE IN GENERAL about the need of conventions
 about keyword usage are very relevant EVEN IF (or actually A FORTIORI if)
 FITS files are used for interchange.
 (I am personally developing a different, more compact, but system dependent
 storage format, but I am keeping as a basic requirement that it MUST be
 one-to-one mappable with FITS : for this reason for instance I am forcing
 all limitations in FITS kewyords, like 8-char max names, etc.
 into my own header kewyords - with binary values )

 Among the points I feel more relevant in Clive Page's mail :

>The software packages noted above fulfil different sets of requirements,
>and data interchange between them is extremely likely.  But there is a
>danger of minor but annoying incompatibilities creeping in unless
>we can agree fairly soon on a few more details of the FITS table format,
>or at least on some conventions for the use of FITS keywords.

 YES, conventions are needed to avoid those incompatibilities, which can
 be very annoying.
 And I would say those conventions are needed beyond the area of databases,
 but, for instance, in the area of keywords describing the content of a
 photon list.
 At the moment I am using provisional, local conventions for the designation
 of columns for a photon list : for instance I call THETA and PHI the angular
 cooordinates of a photon (from the optical axis), X and Y the coordinates in
 micron on the focal plane, and XPOSITN YPOSITN the coordinates in detector
 pixels ...or I call ENERGY true energy in keV, but PHA energy in channels,
 and I ignore whether they are consistent e.g. with EXSAS or PROS - which
 I am told are not compatible among them)
>
>(1) Form of column names.
>contain letters, digits, and underscores.  I would like to propose that
>the first character should not be a digit, since leading digits make it

  I tend to agree

>(2) Column name length.
>There seems no length limit on a column name (except that FITS keyword
>strings cannot be more than 68 chars long). But section 5.3.2.1 says
>that "proper interpretation of the FITS file should not require decoding
>any more than the first eight characters of a character string".  Does

 Usage of readable column names is definitely a bonus, and therefore
 names longer than 8 char could be useful. But in the spirit of "once FITS
 forever FITS" I am ready to live with 8 char.
 However I note that :
 a) FITS binary tables are NOT YET an official standard
 b) the names of the columns (i.e. the VALUE of a TTYPEn keyword) have a
    lower rank than the names of the keywords themselves |
    The keyword names are used by the FITS reader, but the values are
    used by analysis s/w
    Therefore the limitation of 8 chars is appropriate on names like
    TFORMnnn, TTYPEnnn, TUNITnnn but less on their content (and in some
    cases meaningless ... how do you limit to 8 char a VALUE like
    TUNITnnn = 'PHOTON/CM**2/S/KEV' ?

>(3) Sorting.

 I have no comments on this since it is far from the problems I am working
 on at the moment.
 Concerning however keywords of the form Txxxxnnn, which indicate some
 characteristics of the columns, it would be useful to introduce some more
 conventional (optional) keywords.
 At the moment I use TMINnnn and TMAXnnn to indicate the minimum and maximum
 value in column nnn.

>(4) Displaying Sexagesimal Notation.
>Most astronomers still use sexagesimal notation (e.g. 12:34:56,
>-2:34:56.7) for celestial coordinates.
> I wonder if anyone has
>considered an extension to allow the TFORM keyword to specify
>sexagesimal formats?  The obvious way of specifying this is with
>pseudo-fortran forms like "HMSw.d" or "DMSw.d" where the ".d" is needed
>only if the seconds value has a (decimal) fractional part. Obviously
>these formats would have to be handled by special code, but in programs
>written in Fortran-77 (the great majority, I would guess) this is
>already necessary when handling format descriptors such as "Bw.d" or

 I believe here there is a mix-up of two different things.
 BINARY tables are concerned about the representation of the values, therefore
 an angular value shall be best represented as a DOUBLE PRECISION value
 (column of type 1D or nD)
 Sexagesimal notation concerns not internal representation, but displaying
 of the values (TDISPnnn) ... unless one is talking of ASCII tables (which
 I believe will be soon replaced by binary tables)

 Therefore if a binary table COLUMN contains angular quantities, the
 easiest thing is to have them in degrees as a double precision floating
 point value. It is up to the program interpreting them to know how to
 display the numbers.

 The story is different if we talk about header keywords. Here it might
 be worthwhile to define new "types" of keywords for quantities like :
 angles (in degrees), angles (in hours), dates and times

 For instance, in the software I was mentioning at the beginning, I keep
 header information as follows :

 <type> <length> <keyname> <keyvalue>

 where keyname is the FITS-like name (8 char, blank padded), type and
 length are two 1-byte codes, and keyvalue is one or more binary values
 (in general)

 I support basic types like characters, INTEGER*4 (also INTEGER*2 but I am
 going to deprecate it), REAL*4 and REAL*8, but also other derived types
 like ANGLE (which has the same value representation as a REAL*8), and
 perhaps DATE and TIME (for DATE I am inclined to use an INTEGER*4 value
 in seconds since 1/1/1970 which is customary in Unix, while for TIME I
 have not yet made clear my mind).
 The software which uses these files, may use the type field to decide to
 format an ANGLE value in sexagesimal notation, or a DATE value in a
 notation like YYYY-Mon-DD-hh:mm:ss.ff
 To convert my files to FITS I use Bill Pence's FITSIO, and in general
 I use the type field to know which of the FTPKYx routines to call (for
 the basic types)

 Definition of additional kewyord types (for ANGLEs, DATEs and TIMEs) within
 FITS would be useful (and possible, since it will be an addition).

(*) Arrays of keywords

I wonder also whether one should allow somehow arrays of keywords. In my
binary headers the <length> field corresponds to the number of characters
in a string, and is generally 1 for numeric values, but it is possible for
it to be > 1, in which case <value> is a numeric array.
(I am not and I WILL NOT be supporting arrays of strings)
I was reluctant to introduce keyword arrays, but I was moved by a colleague's
request motivated by cases like the following :
let's assume we have an instrument with separate sub-units, like for isntance
7 PMTs (photo-multiplier tubes), of which we want to store the high voltage
setting.
My approach was to have 7 keywords PMTHV1 .... PMTHV7, but he insisted in
having instead a single keyword PMTHV, which is a 7-element array.
I told him I could support this, but regard it as private within his own
software. The main reason is that I do not know of an unique convention
of mapping them to FITS.
For demo purposes at the moment I am proceeding as follows :
I map my PMTHV keyword array to 8 (n+1) FITS keywords

  PMTHV  is an ASCII comment string saying the keyword is an arrau from
         PMTHV1 to PMTHV7
  PMTHV1 is the first element of the array
  ...
  PMTHV7 is the last element of the array

I know for isntance that MIDAS uses array descriptors (the equivalent of
header keywords) and uses a different convention when storing them to FITS.
I am unsure about IRAF.

I wonder whether some convention could be agreed for the representation of
array quantities within FITS headers.

-----------------------------------------------------------------------------
-----------------------------------------------------------------------------
Lucio Chiappetti - IFCTR/CNR Milano        |         U N I C U I Q U E
via Bassini 15 - I-20133 Milano - Italy    |          System  Manager
Internet: LUCIO@IFCTR.MI.CNR.IT            |           FORTUNAE SUAE
Decnet:   IFCTR::LUCIO (39610::LUCIO)      |
Bitnet:   EXOSAT@IMISIAM                   |
-----------------------------------------------------------------------------