🚀 go-pugleaf

I keep seeing folks confused about what is happening
to the heap and gc.  In this note I want to share
some of my observations.  Hopefully we soon
will have much better gc implementations.

First, there are two big levers you have to control the heap.
They are the -ms and -mx switches to Java.  They set the
initial size *in virtual memory* of the heap and the maximum
size in VM for the heap.  Default is 1 and 16 megabytes.

VM is used for two things: handles and objects.  I don't know
the exact fraction, but about 1/4 of the space used in
-ms is for handles (typically).

A handle is 8 bytes (typically).

So, with the defaults there are 1/4 MB of handles, or
1/32 million handles.  There is .75 MB of object space.

As the system runs, it will discover that it is out of
space or that it is a nice time to gc.  It can be
out of space due to lack of handles *or* lack
of space for objects.

When it is really out of space (in its opinion)
it will grow the handle or object space.

Handle space often is grown slowly, so it is possible
to "thrash handles".  If you see this, run with
a bigger -ms to start.  (See below).

Object space is typically grown so that it is
25% free after growth.  If there is not
enough room, many JVM's will fail.

This can appear to be quite early.  The stats
for GC show just object space.  If you have
a 10MB heap, then 2.5 MB of that could be handles.
Of the 7.5 left, after growth the system
wants 25% free.  Thus, it is possible to fail
(OutOfMemoryException) only having used 6MB of heap.

GC is usually run in a single thread.  A multiprocessor
will use just one processor during gc.

Heaps are typically limited to 1GB.  4GB is the
addressing limit for a 32 bit address, and
don't expect this to grow anytime soon.

Running java with -verbosegc tells you many things
about gc.  Unfortunately, you can not get this
information programmatically.  (This is a bug).

So, typical gcs looks like:


<GC: freed 442620 objects, 10849040 bytes in 1531 ms, 54% free
(14658504/26843544)>
  <GC: init&scan: 10 ms, scan handles: 588 ms, sweep: 213 ms, compact: 720 ms>


and 

<GC: managing allocation failure. need 24 bytes, type=1, action=2>
<GC: synchronously running 8366 finalizers>
<GC: not GC'ing classes: 0 verifier running>
<GC: freed 77809 objects, 2799096 bytes in 8501 ms, 14% free
(19366472/131001544)>
  <GC: init&scan: 39 ms, scan handles: 7888 ms, sweep: 574 ms, compact: 0 ms>

In the later, we were trying to allocate a 24 byte object, of which 8 bytes
is probably the header.  GC did a good job and got us back
about 3MB.  It took 8.5 seconds.  After gc, we were 14% free.
The object space was 131MB.  This is not the VM consumed
by the heap, since these numbers (typically) do not
include the handles (bug).

This gc fired off finalization.  This was a synchronous finalization,
thus it ran with the "mutator" (your program) stopped.  You have
to have arranged things that the finalization methods need
no locks held by your program.  This can be hard.

GC elected not to compact.


Another typical thing is:

<GC: managing allocation failure. need 81928 bytes, type=1, action=2>
<GC: managing allocation failure. need 81928 bytes, type=1, action=3>
<GC: managing allocation failure. need 81928 bytes, type=1, action=4>
<GC: expanded object space by 1744896 to 44526792 bytes, 25% free>

We ran out of object space and grew it.

Also:

<GC: managing allocation failure. need 16 bytes, type=2, action=2>
<GC: managing allocation failure. need 16 bytes, type=2, action=3>
<GC: managing allocation failure. need 16 bytes, type=2, action=4>
<GC: expanded handle space by 598016 to 11293488 bytes, 25% free>

We ran out of handle space and grew it.  If the
growth of handle space is small, then you have
to start with a bigger -ms.  Look for:
<GC: expanded handle space by 8192 to 1939864 bytes, 0% free>


And, for grins:

<GC: managing allocation failure. need 120 bytes, type=1, action=2>
<GC: managing allocation failure. need 120 bytes, type=1, action=3>
<GC: managing allocation failure. need 120 bytes, type=1, action=4>
<GC: expanded object space by 17047552 to 110955720 bytes, -13% free>

Don't trust everything you read: you are not -13% free.

--
 ��Դ: ����������BBSվ��bbs.nease.net�E[FROM: www-post@bbs.nease.n]