🚀 go-pugleaf

For a while we've been having problems with running out of address space on
the Explorer.  We'd been assuming that our programs were just consing and
holding onto a lot of data until we noticed an application that ran fine
without TGC but ran out of address space with it.  We started poking around
and discovered that data was being promoted out of generation 2 into
generation 3 when the system automatically collected generation 2.  This was
different from TGC's initial behavior (from rel 3).  What this means is that
data that "survives" (ie. is held onto) for a long time but then dropped can
easily get "tenured" into a generation that isn't collected.

   TI claims that the amount of data mistakenly tenured is very small, but I
think there are cases of applications or usage patterns where this isn't the
case.  Thus, if you've been mysteriously running out of address space would
you try to check <Term> G when you're near the end and see how much data has
survived a generation 2 collection?

   I believe this is a problem, and I believe the problem will get worse if
indicated changes in Rel 6 happen.  The default for
SYS:*GC-MAX-INCREMENTAL-GENERATION* will be reduced from 2 to 1, and the
thresholds for 1 and 2 will be reduced to 20% of RAM and 40% of RAM
respectively.  I think this implies that garbage will accumulate in
generation 2.

   The way the system is now is "unsafe" since a program which at any one
time uses less data than the available address space can run out of address
space.  This may very well be a good tradeoff against performance since in
most cases the amount collected from generation 2 is quite small but it takes
a lot of time to collect, but it is still unsafe.

I would like to see TI do three things:

1. Discuss the issues in the release notes and/or other documentation so that
   users can decide the tradeoff.

2. Add a facility for controlling if the highest automatically collected
   generation, SYS:*GC-MAX-INCREMENTAL-GENERATION*, is automatically promoted
   or not.

3. Allow the user to tune the generation thresholds.  Thus if I have a
   program that cons more than 40% of RAM, holds onto most of it for a while,
   and then drops it, I can tune the system to that without having to resort
   to batch GC.

Please let me know what you think about this issue.

	-- Rich