🚀 go-pugleaf

On Wed, 30 Sep 1992 17:15:10 GMT  <pence@heawk1.gsfc.nasa.gov> said:
>
>Here at the HEASARC we often need to express keyword values which have
>more significant figures than can be stored in a double precision

  I actually wonder how this is SPECIFICALLY related to the example of
  MJD discussed below, or is really general.

>A common example of this problem is the Modified Julian Date
>which has a value something like 48043.8797453703700742

  I am inclined to think this is somehow related to the need of expressing
  time quantities like this.
  What I believe is needed is a way of defining some "compound" or "non-
  primitive" data types. By non-primitive I mean they should go beyond the
  usual CHARACTER, INTEGER, REAL and DOUBLE PRECISION supported by FITS.
  I raised this point already in a previous posting.
  In my view this concerns BOTH header keywords AS WELL AS data quantities
  (e.g. columns in binary tables).
  The sort of data types I am thinking of are quantities like :

  ANGLE   :  an angle in degrees, expressed e.g. as a DOUBLE PRECISION
             floating point number (but liable to be displayed as
             ddd:mm:ss.ff... and/or read as an array of ddd mm ss etc.)
  HANGLE  :  an angle in hours, possibly stored as the previous one,
             but liable to be displayed as hh:mm:ss.ff (with an implicit
             multiplication/division by 15 whenever needed)
  DATE    :  a quantity liable to be displayed as YYYY-Mon-DD (or YYYY-MM-DD
             or DD-MM-YYYY or MM-DD-YYYY according to whatever European or
             US or other form ... I personally prefer YYYY-MM-DD because it
             is simpler to sort). This quantity is likely to be more used
             in headers (e.g. to time stamp a file) than in data areas.
             However it might be worthwhile to represent it natively as
             a numeric quantity on which one can perform arithmetic operations.
             Any choice of a running count of days since an arbitrary zero
             is good, so IAU MJDs are a reasonable choice.
             Since one is interested only at a precision at the day level
             here, an INTEGER*4 is sufficient (using MJD instead of JD is
             preferred)
  DATETIME : a quantity liable to be displayed as YYYY-MM-DD-hh:mm:ss
             that is, similar to the previous one in usage, but with a
             precision at the level of seconds.
             Again more likely to be used in headers than in data areas.
             A running count of seconds since an arbitrary zero, expressed
             as an INTEGER*4, is good. I would recommend using seconds since
             1 Jan 1970 - which is what most Unix systems do - and which
             will cover about 68 years since then.

  TIME :     here is where I am in trouble. I want to express times with
             an arbitrarily high precision (say 1 ms or less). This is
             more the case of a column in a binary table (say, representing
             the time of a bin in a time profile, or the arrival time of a
             photon).
             I am very reluctant to renounce the usage of traditional data
             types, namely either INTEGER*4 or DOUBLE PRECISION.
             One way to cut out the number of significant figures needed is
             therefore to subtract a standard offset from the values.
             I would be personally inclined to use a representation like
             the following :

          -  time in an arbitrary unit (this most likely will be either a
             negative power of 2, 2**-N seconds, in an INTEGER quantity, or
             a decimal fraction like 1 ms, 10 ms, 0.1 ms in a DOUBLE PRECISION
             quantity) since 0 UT of the day containing the first bin or
             first photon of the dataset. This is the quantity stored as
             "data".
          -  a keyword in the header will define the time units (e.g. a
             real number as a fraction of seconds)
          -  another keyword in the header will define the reference time
             corresponding to "0" units, this could be a quantity of type
             DATETIME as above

             In general one does not need the full range of figures when
             doing processing, nor the units to be seconds (if one is doing
             a Fourier transform, one would anyhow subtract the time of the
             first bin, and one would have a power spectrum vs a frequency
             in 1/timeunit instead of Hz), but just when displaying the data
             and in such case one can do the conversion using the info in
             the header keywords

>Currently, we  express this number as 2 separate
>This however has several disadvantages:
>	- it requires 2 keywords = 160 bytes to express the value

  Ah ... sorry Bill. This is the usual "FITS as storage system vs FITS as
  transport system"  ... of course using a native internal representation
  for local storage will allow a more compact representation than FITS
  allows .... while for transport one can gladly spare 160 bytes

>	- it clutters up the header with extra keywords which makes it
>          more difficult for people to read
>In principle there is no reason that this information could not all
>be expressed in a single keyword value, like:
>
>MJD     =      48043.8797453703700742 /  MJD
>
>The only difficultly with this is that there has been no convenient way
>to read or write values in this format, so I have proposed to add (in
>

  The real problem is NOT that there is no convenient way to READ or WRITE
  values in this format, but there is no portable way of STORING them in
  memory (REAL*16 is not standard). That's why I would try the best effort
  to avoid using quantities either than I*4 or R*8.

  The above line of reasoning tries to demonstrate that, at least in the
  area of expressing times and angles, one could do without using "exotic"
  representations.

  One is left however with an important problem, which is at a SEMANTIC level.
  That is, how to know a quantity (a binary table column, or an header
  keyword) is of a specific type (particularly if the type is a nonstandard
  one like DATETIME or ANGLE).
  In a non-FITS system this might be easier (for instance I store all kewyords
  as <type> <length> <name> <values>, and I could code <type> as I wish).
  But I am still left with the problem of expressing them into FITS kewyords
  for export (either a single "hyper-precision" one, like in Bill Pence's
  proposal, or a couple of associated keywords, which is after all not so
  awful).

  The only solution I may think of is of semantic nature, that is to know
  beforehand that a keyword with a particular NAME has a specific TYPE
  associated to it.
  This is already the case e.g. of NAXISn or TFIELDS, which everybody know
  are integers, or of DATE and DATE-OBS which are strings of a particular
  format, or of OBSERVER and ORIGIN, which are strings, etc. etc.
  Definition of additional conventions at this level should remain completely
  within the FITS spirit, and would be highly beneficial to facilitate
  data interchange.
  I would therefore endorse the case that such conventions be defined.

-----------------------------------------------------------------------------
Lucio Chiappetti - IFCTR/CNR Milano        |         U N I C U I Q U E
via Bassini 15 - I-20133 Milano - Italy    |          System  Manager
Internet: LUCIO@IFCTR.MI.CNR.IT            |           FORTUNAE SUAE
Decnet:   IFCTR::LUCIO (39610::LUCIO)      |
Bitnet:   EXOSAT@IMISIAM                   |
-----------------------------------------------------------------------------