🚀 go-pugleaf

I must be dense.  After reading this a couple of times I still don't
understand: how can two Celerons be worse than one when each has its 
own cache?  I don't see how an interrupt serviced by one would cause 
the other's cache to be flushed.

-- Rod

Totally Lost <idletime@my-dejanews.com> wrote:
>The work done by Tomohiro Kawada and others has been interesting.
>I can think of a number of applications where dual/quad overclocked
>Celeron's can produce very cost effective clusters for number crunching.
>The crypt cracking teams will get a great windfall from this work.

>I do performance studies and optimization for a living. One of the
>biggest problems people have porting applications from big iron
>SMP servers to X86 SMP servers is memory foot print and working
>set locality problems which make X86 SMP scaling difficult at best.

>Big Iron SMP servers typically have larger L1/L2 caches with
>a much faster memory subsystem under them. The size of the
>working set directly predicts the performance of most
>applications. To help clients understand this I often use the following
>visual aid in my presentations, which is the output of a tool
>I use to measure the performance of their system as the working
>set size is increased:

>Counts are thousands of MemOps/second.
>SunOS svr20 5.5.1 Generic sun4m sparc SUNW,SPARCstation-20

>Set  Sequential Random
>Size  Cnt   Pct Cnt   Pct
>----- --------- --------- 0%        25%        50%        75%       100
>   2K 29685 100 21271 100 |          |          |         *|          @
>   3K 29657 100 21244 100 |                              *            @
>   4K 29652 100 21231 100 |          |          |        * |          @
>   6K 29644 100 21264 100 |                               *           @
>   8K 29652 100 21203 100 |          |          |        * |          @
>  12K 29644 100 21245 100 |                              *            @
>  16K 29625 100 21221 100 |          |          |        * |          @
>  24K 10996  37 18120  85 |               @          *
>  32K  7943  27 16190  76 |          |@         | *        |          |
>  48K  7941  27 13716  64 |           @       *
>  64K  7939  27 12067  57 |          |@     *   |          |          |
>  96K  7943  27 10802  51 |           @   *
> 128K  7947  27  9877  46 |          |@  *      |          |          |
> 192K  7946  27  9837  46 |           @  *
> 256K  7909  27  9828  46 |          |@  *      |          |          |
> 384K  7773  26  9806  46 |           @  *
> 512K  7652  26  9766  46 |          @  *       |          |          |
> 768K  7619  26  9765  46 |          @  *
>1024K  7580  26  9755  46 |          @  *       |          |          |
>1536K  2164   7  9642  45 |  @          *
>   2M  2045   7  9634  45 |  @       |  *       |          |          |
>   3M  2044   7  9602  45 |  @          *
>   4M  2041   7  9595  45 |  @       |  *       |          |          |
>   6M  2038   7  9592  45 |  @          *
>   8M  2042   7  8823  41 |  @       | *        |          |          |
>  12M  2035   7  8720  41 |  @         *
>  16M  1863   6  8723  41 |  @       | *        |          |          |
>  24M  2023   7  8596  40 |  @         *
>  32M  1657   6  9590  45 | @        |  *       |          |          |
>  48M    10   0  8040  38 @           *
>  64M    40   0  6552  31 @         *|          |          |          |

>The graph is plotted with all values relative to the first sequential
>value. The test reflects the performance losses due to the working
>set size impact on the VM/Cache/Memory subsystems. This includes
>TLB misses, cache misses and memory latency. Cache design and memory
>allocation coloring present very different responses to this test.
>Poorly colored memory with direct mapped caches shows up with early
>degradation slopes, while good coloring produces sharp fall-offs at
>each cache boundry.

>This particular machine didn't have a lot of
>memory and started paging above 32M, with 32K L1 and 1024K L2 cache
>sizes. Increasing the working set by a single cache line
>often reduces the performance to the next larger (and slower)
>memory object.

>In a real system, there are many sources that scale into the working
>set size(s). The size of the active interrupt service routines and
>OS scheduler are often near the size of L1/L2 caches on many X86
>systems, thus each interrupt flushes the cache and effectively runs
>at memory or L2 cache speeds. In an SMP system, both CPU's have to
>share the memory performance. If the L1/L2 caches are too small, then
>two processors are worse than one, because they will flush each other
>with distributed interrupts. The cost of flushing the cache by context
>switches and interrupts is two 2X ... memory cycles required to bring
>in the new context/interrupt plus the memory cycles required to replace
>the active process memory on exit. One reason that X86 SMP servers
>may scale, is that network interrupt loads that flush a uniprocessor,
>only impact one processor in a multiprocessor system ... especially
>if the interrupts are bound to a single processor. Ditto for the disk
>subsystems. SCSI controllers that have high interrupt loading can
>kill an SMP server if the interrupts are distributed. NCR825's are
>a very good example when used with very poor SCSI scripts and drivers.

>Understanding the locality and working set sizes, some X86 vendors
>have post processed their kernel builds to improve locality and
>placement. The work done by the SCO Unixware team on the object code
>segments has produced remarkable results. These vendors often have
>large kernels and feature sets, with same/better overall performance.

>Celron's with their 128K L2 cache can easily produce worse performance
>under modest to heavy load than a single processor in an SMP system.
>In many server systems, even dual PII's with 512K L2 caches fail to
>scale or produce negative results under modest to heavy work loads.
>In some applications, even Xenon's large caches fail to scale or
>produce negative results in comparision to uniprocessors of the same
>architecture. Simply put, it's not uncommon to find applications where
>two processors trashing the caches with SMP interrupts or shared memory
>produce much worse performance than the same system with only a single
>processor enabled.

>Linux is not well multi-threaded, and has a pretty large foot print
>with very little locality in either the OS or it's primary applications.
>I would expect that for many users, dual Celeron's may have a negative
>performance impact much of the time.

>To observe that Dual Celron's produce an observed improvement under
>certain lightly loading applications may be true. To assert that
>they scale under modest to heavy work loads is probably a huge mistake.
>To expect that they will scale well under Linux with heavy loads is
>almost certainly folly.

>John Bass
>UNIX Systems Consultant


-- Rod
----------------------------------------------------------------------
Sunset Systems                           Preconfigured Linux Computers
http://www.sunsetsystems.com/                      and Custom Software
----------------------------------------------------------------------