🚀 go-pugleaf

Draft OS/2 Font FAQ
Copyright (C) 1993 by David J. Birnbaum
All right reserved

4 June 1993 

A couple of weeks ago I posted an inquiry to comp.fonts,
comp.os.os2.misc, and the OS2-L ListServ concerning some apparent
peculiarities in the way OS/2 handles font files. These
"peculiarities" actually reflect regular, systematic differences
in OS/2, Windows, and DOS font handling, which are not
conveniently described in end-user documentation. This posting is
intended to spare others some of the confusion I encountered as a
result of this paradigm shift.

This is the first (draft) distribution of this document and
corrections and suggestions are welcome. I am grateful to Henry
Churchyard, Marc L. Cohen, Bur Davis and Kamal Mansour for
helpful discussions; they are not, of course, responsible for any
misinterpretation I may have inflicted on their comments.

Preliminaries
=============

Character: an informational unit consisting of a value (usually a
byte) and roughly corresponding to what we think of as letters,
numbers, punctuation, etc.

Glyph: a presentational unit corresponding roughly to what we
think of as letters, numbers, punctuation, etc.

Character vs glyph: Glyph and character are not necessarily the
same; the character <a> may be mapped to a Times Roman Lower Case
<a> glyph in one font and to a Helvetica Lower Case <a> glyph in
another font. Change of glyphs normally means a change in style
of presentation, while change in characters normally means a
change in information. There are gray areas and the definitions
provided above are general, approximate, and imprecise.

Character set: an inventory of characters with certain assigned
values. ASCII is a 7-bit character set that specifies which "character
cell" (byte value) corresponds to which informational unit.

Code Page: essentially synonymous with character set.

Font: A collection of glyphs. A specific font may be isomorphic
with a specific character set, containing only glyphs
corresponding to characters in that set, with these glyphs mapped
to the same byte values as the characters they are intended to
represent. PostScript fonts often contain additional (unmapped)
characters. Most importantly, PostScript fonts may sometimes be
remapped by an operating environment, which is what leads to the
disorienting cross-environment mismatch that spurred my original
posting.

Fonts may be bitmapped or outline in format; a bitmapped format
corresponds to a particular size and weight for a particular device or
device resolution, while a single outline font is used to generate
multiple sizes as needed. Within an outline font system, different
weights (bold, semibold, italic, etc.) may be encoded as separate font
resources (separate outline files used to generate the glyphs) or may
all be generated from a single outline (slanting characters to make
"italics," fattening them for "bold," etc.).

Fonts Under Dos
===============

I used a large assortment of fonts under DOS for intricate
multilingual work. My setup at that time consisted of a library
of bitmapped fonts that could be sent to my HP LaserJet II
printer, as well as a set of fixed-size, fixed-width screen fonts
that were supported by my Hercules Graphics Card Plus (not the
same as Hercules Graphics; the "Plus" included an ability to
store 3072 screen glyphs and display any of these together, while
standard character-mode displays were normally limited to 256 or
512 such entities).

Using XyWrite as a word processor, I would enter a "Mode" command
to change fonts and character sets simultaneously; this would
make different sets of screen glyphs available at the keyboard
and would insert a font-change command for my printer into the
text stream. The "Mode" and font-change commands were not
displayed on the screen. The result was not WYSIWYG, since I was
limited to fixed-width screen display and since I had far more
printer glyphs available than the 3072 limit imposed by my video
card; I used a brightness attribute to indicate bold, I used the
same screen font for different sizes of printer fonts, etc. This
worked and worked well, in that I could see (for example)
Russian, Greek, English, Polish, and other characters
simultaneously on the screen and I could print documents
combining them.

Architecturally, what was going on was that the character sets
(code pages) and fonts were entirely isomorphic and were hard-
coded. If I put a particular Russian letter into cell 246 of my
screen and printer fonts, that character was always there, and
any strategy that would let me access this cell (remapped
keyboards, numeric keypad) was guaranteed always to find the same
character.

Windows
=======

I recently began using PostScript fonts in Windows with AmiPro as
my word processor. These fonts came with printed cards
indicating the glyph mappings; I could look at the card and it
would tell me that a specific character lived in cell 246, and if
I entered Alt-0246 at the numeric keypad that glyph would appear
on the screen. If I loaded the font into Fontographer for
Windows, these glyphs would be arrayed in cells according to the
map provided by Adobe with the fonts. Fontographer also revealed
that these fonts had other, "unmapped" glyphs assigned to cells
above 255.

Given what appeared to be a hard correspondence among what I saw
in Fontographer, what was printed in Adobe's maps, and what was
displayed when I entered something at the keyboard, I naively
assumed that PostScript fonts were operating much like my
bitmapped fonts under DOS. There were some obvious differences,
the primary one being that glyphs of different sizes were all
drawn from the same font resource files under PostScript, but it
appeared as if a glyph lived in a certain cell.

Differences between Windows and OS/2
=====================================

This assumption was incorrect; PostScript fonts can be subdivided
into two types, one of which observes hard and invariant
encodings similar to those that apply to my bitmapped fonts,
while the other represents a completely different font mapping
strategy. This difference became apparent only when I attempted
to share PostScript fonts between Windows and OS/2 and got some
unexpected results.

A PostScript font under Windows involves two files, a PFB (PostScript
Font Binary) file, which contains the PostScript instructions needed to
draw each glyph and some mapping information, and a PFM (Printer Font
Metrics) file, which encodes width and kerning information. A
PostScript font under OS/2 also uses the same PFB file, but instead of
the PFM file it uses an AFM (Adobe Font Metrics) file. The AFM and PFM
files contain much of the same basic information (although the AFM file
is somewhat more complete); the most important differences are in
format (AFM is plain text, PFM is binary) and use (OS/2 uses AFM,
Windows uses PFM).

Installation under Windows and Win-OS/2
=======================================

The OS/2 2.0 Font Palette tool (see below for changes to be
introduced with 2.1) by default installs fonts (both PFB and AFM
files) into the \os2\dll directory. Win-OS/2 by default installs
PFB files into \psfonts and PFM files into \psfonts\pfm. These
defaults can be changed; since OS/2 and Win-OS/2 use the same PFB
files, the user can save disk space by allowing these to be
shared (through installing into the same directory, e.g., install
OS/2 fonts into the \psfonts directory instead of \os2\dll.) Note
that fonts must be intalled and removed through the Font Palette;
if you copy, move, or delete a font file without using the Font
Palette, the system configuration files are not updated and all
hell breaks loose.

Deleting fonts from Win-OS/2 causes the system to update the win.ini
file to remove references to the font, but does not delete any files
physically. Deleting fonts from the OS/2 Font Palette updates the
os2.ini configuration file _and_ physically deletes the AFM and PFB
files from the disk.  This means that if you are sharing PFB files
between OS/2 and Win-OS/2, you can delete a Win-OS/2 font without
hurting native OS/2 operations, since the PFB reamins installed where
OS/2 thinks it is. But if you delete an OS/2 font using the Font
Palette, the PFB file is erased from the disk even though the win.ini
file is not updated, so that Win-OS/2 thinks it is still there.

FontSpecific PostScript Encoding
================================

Every PFB file contains an "encoding vector"; this is a plain
text line embedded near the head of the PFB file. Encoding
vectors are of two types: AdobeStandardEncoding and everything
else. Adobe usually uses the label "FontSpecific" for fonts that
are not encoded according to AdobeStandardEncoding, and I use it
as a cover term here for any such font.

If you look at the readable plain text information at the head of
a FontSpecific type font, it includes a range of text that
begins:

	/Encoding 256 array

followed by a bunch of lines, each of which includes a number
(which corresponds to a cell in the font layout) and the name of
the glyph that lives in that cell. The unreadable binary data
below this array specification lists the name of each glyph and
the PostScript instructions for how the glyph is to be drawn.
There may be PostScript code for drawing glyphs that are not
included in the mapping array, but only glyphs mentioned in the
array specification are available to applications.

FontSpecific type fonts _are_ comparable to the bitmapped fonts I
used under DOS. Each character physically is assigned to a
specific cell within the font file and operating environments are
not allowed to remap these. The glyph in cell 246 will be the
same in both Windows and OS/2.

AdobeStandardEncoding
=====================

AdobeStandardEncoding is a specific mapping of certain glyphs to
certain cells; in this respect it resembles FontSpecific
encoding. Because it is standardized, the array is not spelled
out in the PFB file; the line

	/Encoding StandardEncoding def

tells Adobe Type Manager (ATM, either the Windows and Win-OS/2
version or the native OS/2 version) that the encoding is
"standard," and the environments are expected to know what this
standard is without having the array spelled out in each font
file.

Although AdobeStandardEncoding is a real mapping, there is an
importance difference between it and various FontSpecific
mappings: operating environments are expected to remap
AdobeStandardEncoding fonts according to their own requirements.
That is, although AdobeStandardEncoding does assign glyphs to
cells, no operating environment actually uses these assignments
and any environment remaps the glyphs before rendering them.
Confusion arises because Windows and OS/2 remap such fonts in
different ways.

AdobeStandardEncoding under Windows (and Win-OS/2)
==================================================

An AdobeStandardEncoding font under Windows is remapped according
to a character map (code page) that MicroSoft calls Windows ANSI
(can other code pages be installed in Windows?). This determines
which character resides in which cell and the font is remapped so
that glyphs and characters will correspond. Since Fontographer
for Windows is a Windows application, it displays glyphs not in
the cells in which they live according to AdobeStandardEncoding,
but in the cells to which they get reassigned under the remapping
to Windows ANSI. There is nothing explicit in the PFB file that
associates these characters with the specific cells in which they
appear under Windows.

AdobeStandardEncoding under OS/2
================================

OS/2 operates within a set of supported code pages; two system-
wide code pages are specified in the config.sys file and an
application is allowed to switch the active code page to any
supported code page (not just these two). DeScribe, for example,
currently operates in code page (CP) 850, which includes most
letters needed for western European Latin alphabet writing. CP
850 does not contain typographic quotes, en- and em-dashes, and
other useful characters. It does contain the IBM
"pseudographics," which are useful for drawing boxes and lines
with monospaced fonts.

When the user inputs a value (through the regular keyboard or the
numeric keypad), the application checks the active CP, looks up
in an internal table the name of the character that lives in that
cell within that CP, and translates it into a unique number that
corresponds to one of the 383 glyphs supported by OS/2 (the union
of all supported code pages). This number is passed to PM-ATM
(the OS/2 ATM implementation), which translate the glyph number
into the glyph name that PostScript fonts expect and searches the
font for that name. The system never looks at where a glyph is
assigned under the AdobeStandardEncoding array; rather, it scans
the font looking for the character by name and gives it an
assignment derived from the active code page. This is the
remapping that OS/2 performs on AdobeStandardEncoding type fonts.

As a result, a situation arises where, for example, <o+diaeresis>
is mapped to cell 246 under Windows ANSI but to cell 148 under CP
850. Using the identical PFB file, this glyph is accessed
differently in the two operating environments.

Consequences for OS/2 users
===========================

If your font has a FontSpecific encoding, there are no unexpected
consequences; the same glyphs will show up at the same locations
in both Windows (Win-OS/2) and native OS/2. Regardless of what
the active code page is, if the font has a FontSpecific encoding
OS/2 goes by cell value; a specific glyph is hard-coded to a
specific cell and OS/2 will give you whatever it finds there,
even if what it finds disagrees with what the active code page
would normally predict. In other words, FontSpecific encoding
means "ignore the mapping of the active code page and rely on the
mapping hard-coded into the font instead."

If your font has an AdobeStandardEncoding encoding, the following
details obtain:

1) The same PFB file may have glyphs that are accessible in one
environment but not another. For example, if DeScribe thinks it
is operating in CP 850, there is no access to typographic quotes,
even if those do occur in the PFB file and even if Windows can
find them in the same exact font file. DeScribe could switch code
pages, but if the application isn't set up to do so (and DeScribe
currently isn't), those characters are absolutely inaccessible to
the user.

2) If the active code page includes a character that isn't
present in the font, OS/2 has to improvise. For example,
AdobeStandardEncoding fonts do not normally include the IBM
pseudographics, yet the user who inputs the character value for
one of these sends the system off to look for it. As described
above, OS/2 first checks the active font for the glyph name that
corresponds to that character and, if it finds it, displays it.
If the glyph isn't found, OS/2 looks to the system Symbol font.
This is not reported back to the user in DeScribe; if I have
Adobe Minion active (AdobeStandardEncoding, no information
anywhere in the font files for pseudographics) and input a
pseudographic character, DeScribe tells me it is still using
Adobe Minion, even though it has fetched the character it
displays and prints from the Symbol font, a different font
resource file.

Advice to the User
==================

OS/2's code page orientation provides some advantages, in that it
separates the character set (code page) mapping from the encoded
font mapping. The main inconvenience isn't a loss of function,
but a disorientation as users become accustomed to the new
paradigm.

If you need a glyph that you know is in your PFB file but that
isn't in the active code page (and if you can't change code pages
within your application), you can't get at it in OS/2 without
tampering with the font files. To tamper, you can use font
manipulation tools to redesignate the PFB file as FontSpecific
("Symbol" character set to Fontographer). If you then map the
glyphs you need into one of the lower 256 cells (with some
limitations), they will be accessible in all environments. The
Fontographer manual does not explain what the "Symbol" character
encoding label really does, it just tells you not to use it
except for real symbol fonts. In fact you should use it for any
font that will not correspond in inventory to the code page
supported by your application, which means any non-Latin fonts.

You do not have to recode all your fonts, and you wouldn't normally
want to do so, since Fontographer hinting is not nearly as good as
Adobe's own hand-tuning and regenerating a font regenerates the hints.
All you have to do is make sure you have one FontSpecific type font
installed that includes your typographic quotes, etc. for each typeface
you need. Within DeScribe, you can then write a macro that will let you
switch fonts, fetch a character, and switch back, thereby allowing you
to augment any group of fonts with a single, shared set of typographic
quotes (or whatever) that you put in a single FontSpecific font.
Alternatively, OS/2 also supports CP 1004, which does contain
typographic quotes and other characters used for high-quality
typography, but the user may not be able to convince an application to
invoke this code page if it was not designed to do so.

You can have any number of FontSpecific fonts installed, which
means that there is a mechanism for dealing with unsupported
character sets (code pages).

You can also tinker with the font files to try to trick the
operating system. For example, using Fontographer or other
utilities, you can change the name assigned to a glyph
description within the PFB file. If you want to use
AdobeStandardEncoding and you want to see a specific glyph at a
specific cell when DeScribe thinks it's using CP 850, you have to
make sure that the name assigned to the description of that glyph
is what DeScribe expects to find. OS/2 doesn't care whether, say,
<o+diaeresis> really looks like <o> with two dots over it, as
long as it bears the right name.

This second approach is obviously far more complex and provides
much more opportunity for error. Its advantage is that OS/2 does
not support case conversion and sorting (other than in machine
order) for unsupported code pages, since these operations depend
on character names. Keeping supported names from supported code
pages while changing the artwork is one way to maintain order and
case correspondences while increasing the range of glyphs
actually supported. I have not experimented with this approach,
since the use I would get out of the adding functionality (over
the FontSpecific encoding approach) is not worth the amount of
effort required.

OS/2 2.1 and beyond
===================

OS/2 2.1 will change some aspects of font handling. First, OS/2
2.0 GA+SP has a bug that can cause OS/2 to crash when an AFM file
with more than 512 kern pairs is read. This is fixed in 2.1.
(This bug is separate from a design limitation in MicroSoft
Windows that causes large kern tables to be read incorrectly.
This problem is still under investigation; watch this space for a
report.)

Fonts in 2.1 will be installed by default into the \psfonts
directory, so that they will normally be shared with Win-OS/2
fonts. (The user will still be able to specify a directory; all
that will change is the default). The user will also be able to
instruct the Font Palette not to delete font files when fonts are
uninstalled, so as to avoid clobbering a Win-OS/2 font by
removing it from native OS/2 use through the Font Palette
(although the default will still be to delete the physical font
files).

OS/2 will stop using AFM files and will replace these with OFM
files, a binary metrics file (different from PFM) that OS/2 will
compile from the AFM file during font installation. This will
speed font loading, since the system will not have to parse a
plain text metrics file. Additionally, the OS/2 PostScript
printer driver used to install its own, large font files, but
will now use the OFM and PFB files, thereby saving 50k-200k of
disk space per installed font outline.

IBM's long-term goal is to replace the 383-entity inventory of
supported glyphs with Unicode. This is very much a long-term goal
and there is not even a hint of when it might become available.
It has its own problems, stemming from the fact that Unicode is
essentially a character standard and glyph and character
inventories may differ is assorted ways, but it will be a
significant step in the proverbial right direction.


--
Professor David J. Birnbaum         djbpitt+@pitt.edu [Internet]
The Royal York Apartments, #802     djbpitt@pittvms   [Bitnet]
3955 Bigelow Boulevard              voice: 1-412-687-4653
Pittsburgh, PA  15213  USA          fax:   1-412-624-9714