🚀 go-pugleaf

"Ted Tripp" <ttripp@skypoint.com> wrote in message
news:3e6deb3f@newsgroups.borland.com...
> The BCB5 manual recommends no more than 16 threads
> per processor.

Ignore it.  It's rubbish.  I've seen programs running hundreds of threads at
a time without any problems.

> my app usually runs the CPU at about 100%, is there a
> way to control that?

That is typically caused by your code running tight loops that don't yield
control to other threads/processes.  Try introducing Sleep() calls in your
loops.


Gambit

"Remy Lebeau (TeamB)" <gambit47@yahoo.com> wrote in message
news:3e6e2fb9@newsgroups.borland.com...
>
> "Ted Tripp" <ttripp@skypoint.com> wrote in message
> news:3e6deb3f@newsgroups.borland.com...
> > The BCB5 manual recommends no more than 16 threads
> > per processor.
>
> Ignore it.  It's rubbish.  I've seen programs running hundreds of threads
at
> a time without any problems.
>
> > my app usually runs the CPU at about 100%, is there a
> > way to control that?
>
> That is typically caused by your code running tight loops that don't yield
> control to other threads/processes.  Try introducing Sleep() calls in your
> loops.

Of course, only one thread at a time is actually executing at any given
moment (one thread per processor).  It is also possible that having too many
(active) threads can have potentially devestating effects on performance due
to the overhead required to perform a context switch.  I seem to recall
reading something once which talked about the optimum number of simultaneous
threads per processor being somewhere around 3 or 4.  Less than that, and
you are wasting processor time (assuming the threads go to sleep once in a
while); more than that, and your process actually begins to slow down (if
all of the threads are active at the same time) because of context
switching.

I'll have to try to find where I read that...or perhaps someone else
remembers the same article(?) and knows where it is.  WDJ?  CUJ?

- Dennis

"Dennis Jones" <djones@nospam.com> wrote in message
news:3e6f8ce5$1@newsgroups.borland.com...

> Of course, only one thread at a time is actually executing at
> any given moment (one thread per processor).

Yup.  As such, threads do need to periodically yield their time slices to
other threads/processes in order to keep things rolling smoothly.

> It is also possible that having too many (active) threads can have
> potentially devestating effects on performance due to the
> overhead required to perform a context switch.

Potentially, though I don't really run across that problem very often.

I've also been hearing more leately about fibers, which allow multiple tasks
to be performed by a single thread, and the thread manually controls the
context switching between them instead of the OS.

> I seem to recall reading something once which talked about the
> optimum number of simultaneous threads per processor being
> somewhere around 3 or 4.

Yuk, way too small for today's applications.

> more than that, and your process actually begins to slow
> down (if all of the threads are active at the same time)
> because of context switching.

That can be a matter of debate.  For example, my company's software can
potentially run upwards of at least 100 threads at any given time (depending
on the size of the hardware system it is running on and managing).  Every
thread in of itself is critical, monitoring individual hardware pieces in
real-time, and we haven't notice any significant slowdowns from context
switching between them.


Gambit

On Wed, 12 Mar 2003 11:39:18 -0800, "Dennis Jones" <djones@nospam.com> wrote:

>I seem to recall
>reading something once which talked about the optimum number of simultaneous
>threads per processor being somewhere around 3 or 4.  Less than that, and
>you are wasting processor time (assuming the threads go to sleep once in a
>while); more than that, and your process actually begins to slow down (if
>all of the threads are active at the same time) because of context
>switching.

I would tend to think this is old time info maybe valid
on 286 machines or some other CPU with really
clunky context switching(as in simulating rebooting
the machine!)  4 threads/cpu?  Man! You couldn't 
even bring up the Windows desktop!

I think a Pentium can handle a few more than
that. :)


-- 

"Only choice is an oxymoron."

"Michael Kelly" <mkelly00@netzero.net> wrote in message
news:hq9v6vkjgup69jju5tdj5ltp9s72su9vlq@4ax.com...
> On Wed, 12 Mar 2003 11:39:18 -0800, "Dennis Jones" <djones@nospam.com>
wrote:
>
> >I seem to recall
> >reading something once which talked about the optimum number of
simultaneous
> >threads per processor being somewhere around 3 or 4.  Less than that, and
> >you are wasting processor time (assuming the threads go to sleep once in
a
> >while); more than that, and your process actually begins to slow down (if
> >all of the threads are active at the same time) because of context
> >switching.
>
> I would tend to think this is old time info maybe valid
> on 286 machines or some other CPU with really
> clunky context switching(as in simulating rebooting
> the machine!)  4 threads/cpu?  Man! You couldn't
> even bring up the Windows desktop!
>
> I think a Pentium can handle a few more than
> that. :)

Of course, I'm not necessarily agreeing with, or advocating, such a severe
limitation.  I personally have applications that routinely use 10-20 (or
more) threads (although not all of them are necessarily active at any given
time).  And since I don't remember the source of the information I've
paraphrased, I can't say with any certainty whether it was written with any
particular processor in mind.  Although I'm pretty sure was written long
after the pentium processor was widely available.

It may very well be that you can have 100's or even 1000's of threads
operating siumultaneously and that it won't affect the performance of your
application.  But it really does depend on the what the threads are being
asked to do.  If they are spending the bulk of their time sitting around
waiting for something else to happen (semaphore, event, critical section,
etc), then certainly, you can have that many threads and never take a
performance hit because the OS will efficiently avoid switching into the
context of a thread that is waiting for an OS event.  On the other hand, if
every one of those threads is busy doing something (or worse, is in a tight
loop watching the state of a flag), then the amount of context switching
would probably be significantly measureable, and seriously affecting
performance.

I would be interested in seeing an analysis of performance gain vs. context
switching overhead using varying numbers of threads for a given application.
Of course the optimum number of threads will differ depending on the type of
application, the role of each thread, and whether the threads are usually
busy, or usually waiting, but I am quite sure there will be some point at
which adding more threads becomes counter-productive -- for any application

- Dennis

"Dennis Jones" <djones@nospam.com> wrote in message
news:3e6fb851$1@newsgroups.borland.com...
> On the other hand, if every one of those threads is busy
> doing something (or worse, is in a tight loop watching the
> state of a flag), then the amount of context switching
> would probably be significantly measureable, and
> seriously affecting performance.

I can only speak for my own software in that regard, but I don't see that
happening.  In my company's primary software, the one running 100s of
threads at a time, the majority of them are actively processing live data
through the lifetime of each thread from start to stop without much in the
way of noticable performance hits.


Gambit

"Remy Lebeau (TeamB)" <gambit47@yahoo.com> wrote in message
news:3e6fc46e$1@newsgroups.borland.com...
>
> "Dennis Jones" <djones@nospam.com> wrote in message
> news:3e6fb851$1@newsgroups.borland.com...
> > On the other hand, if every one of those threads is busy
> > doing something (or worse, is in a tight loop watching the
> > state of a flag), then the amount of context switching
> > would probably be significantly measureable, and
> > seriously affecting performance.
>
> I can only speak for my own software in that regard, but I don't see that
> happening.  In my company's primary software, the one running 100s of
> threads at a time, the majority of them are actively processing live data
> through the lifetime of each thread from start to stop without much in the
> way of noticable performance hits.

Are you saying that the program experiences no degredation in performance
due to those 100's of context switches, or that the program is not
significantly impaired by 100's of context switches?  Saying that the
program still performs well, or within expected limits, or even that you
don't notice any degredation in performance, doesn't necessarily mean that
it isn't happening.  However, you certainly can't complain too much about it
if the program still performs well!

You probably have a case where fewer threads might be more efficient from
the OS'es perspective (because of context switching), but would be
unacceptable from the application's perspective (because the application is
expected to, or required to, perform those 100's of operations
simultaneously).

Is it possible for your application to operate with fewer threads (say, 4 or
5)?  I would be curious to see the results of measurements where the
application runs with significantly fewer threads, and then again with
significantly more threads (where both cases perform the same amount of
work).

- Dennis

I suspect the comment about limiting the number of threads was good advice
back in the days of the 386, 486 and NT <= 3.51 but doesn't apply these
days.  I used to solve people's problems and see them come back with a
"thank you" back then when I told them to cut back on the number of threads.
Those problems don't seem to be an issue any more.

.  Ed

> Remy Lebeau wrote in message
> news:3e6e2fb9@newsgroups.borland.com...
>
>
> Ignore it.  It's rubbish.  I've seen programs running
> hundreds of threads at a time without any problems.

"Dennis Jones" <djones@nospam.com> wrote in message
news:3e6fd97b$1@newsgroups.borland.com...

> Are you saying that the program experiences no degredation
> in performance due to those 100's of context switches, or that
> the program is not significantly impaired by 100's of context
> switches?

Both, actually.

> You probably have a case where fewer threads might be
> more efficient from the OS'es perspective (because of
> context switching), but would be unacceptable from the
> application's perspective (because the application is
> expected to, or required to, perform those 100's of
> operations simultaneously).

Most of those threads are doing the exact same thing (processing the same
kind of data, just from different sources), so I don't really see how one
can be seen as more efficient as another from the OS's perspective

> Is it possible for your application to operate with fewer
> threads (say, 4 or 5)?

No, it is not (ithout a major if not complete re-write of the app, which is
in the millions of lines of code now).  Each thread is tied to a particular
source of incoming data from various hardware.  Because of the nature of the
data (real-time live feed), even trying to merge two of those threads into a
single thread to handle both sources would probably degrade both operations
from the application's point of view.  Even if it didn't, and we merged them
2 at a time, we'd still need much more than just 4-5, we'd still need closer
to 30-40.


Gambit

Thanks for the info folks.  Yes, it is true I do have plenty of looping.
This is a desktop (so far) app that does multiple http(Indy 9) reads
simultaneously.  It seem to rock the first 10-20 seconds then slows down--I
can see the CPU avail disappear.  I would have thought the reads on my
clunky DSL would act as a built in sleep (though, of course we can assume
the customer may have something better than a clunky DSL).  Anyway, I'll try
introducing a few Sleeps within the next few days and see if I can get some
performance back.  Oddly, I seem to be having more difficulty simply getting
the OnWork Bytes/Sec to work properly, but that's a post for a different ng.

Another question, I vaguely recall reading that threads are initialized with
a certain (default?) amount of memory.  I seem to recall something like
1MB--maybe this was a nonwindows OS...  Anyway, if true, can this, should
this, be tweaked?  If not true, NM.

Again, thanks for all the info...

ttripp

"Ted Tripp" <ttripp@skypoint.com> wrote in message
news:3e70406f@newsgroups.borland.com...
> Another question, I vaguely recall reading that threads are initialized
with
> a certain (default?) amount of memory.  I seem to recall something like
> 1MB--maybe this was a nonwindows OS...  Anyway, if true, can this, should
> this, be tweaked?  If not true, NM.

This is not true.  A thread is not initialized with a particular amount of
memory.  Any thread can allocate as much memory from the global heap as it
needs to (assuming there is memory available).

The Windows API also provides something called "thread local storage" (TLS)
which is memory that can be allocated by a thread and is known only to the
thread that allocated it.  But here again, there is no pre-set amount that
is made available to the thread.

- Dennis

On Thu, 13 Mar 2003 02:27:02 -0600, "Ted Tripp" <ttripp@skypoint.com> wrote:

>Another question, I vaguely recall reading that threads are initialized with
>a certain (default?) amount of memory.  I seem to recall something like
>1MB--maybe this was a nonwindows OS...  Anyway, if true, can this, should
>this, be tweaked?  If not true, NM.

Perhaps you are thinking in Windows the main thread uses
a default heap that is also used by Windows.  So, one possible
optimization is to allocate a heap to a thread so that only that
thread is accessing it.  I seem to remember the 1 MB myself
but I don't know if it's a Windows default or has to do with
the linker used to link the app.

-- 

"Only choice is an oxymoron."

On Thu, 13 Mar 2003 02:27:02 -0600, "Ted Tripp" <ttripp@skypoint.com> wrote:

>Another question, I vaguely recall reading that threads are initialized with
>a certain (default?) amount of memory.  I seem to recall something like
>1MB--maybe this was a nonwindows OS...  Anyway, if true, can this, should
>this, be tweaked?  If not true, NM.

Perhaps you are thinking in Windows the main thread uses
a default heap that is also used by Windows.  So, one possible
optimization is to allocate a heap to a thread so that only that
thread is accessing it.  I seem to remember the 1 MB myself
but I don't know if it's a Windows default or has to do with
the linker used to link the app.

-- 

"Only choice is an oxymoron."

"Dennis Jones" wrote:
> performance hit because the OS will efficiently avoid switching into the
> context of a thread that is waiting for an OS event.  On the other hand,
if
> every one of those threads is busy doing something (or worse, is in a
tight
> loop watching the state of a flag), then the amount of context switching
> would probably be significantly measureable, and seriously affecting
> performance.

I have tried to write a program to measure this, but it is quite tricky. I
will post the code at the bottom. Use commandline bcc32 to compile it.
Warning: very not userfriendly, use at own risk, etc.

Program takes 2 parameters:
threads #numthreads #delaycounter

#numthreads = the number of threads the program creates before beginning
#delaycounter = a number that is used to slow things down. 1000 is approx 1
sec on my computer, 750MHz.

I don't know if a conclusion can be drawn, but 2024 is max threads for my
machine and the program runs about 40 percent slower. 1 thread takes 1040
msec, 2024 threads take 1412 msec to complete.

I have tried to measure only the difference context switching, but I don't
know how succesful I was.

HTH,

Micha.


---code threads.cpp---

#define WIN32_LEAN_AND_MEAN

#include <windows.h>
#include <stdio.h>
#include <stdlib.h>

int num_iterations;
bool active;

unsigned long __stdcall thread_func(void* data)
{
unsigned __int64 i;

while (!active) {}
for (i=0; i < (*reinterpret_cast<unsigned __int64*>(data)); i++);

return NULL;
}

int main(int argc, char** argv)
{
HANDLE* tid;
int i, j, numthreads;
unsigned __int64 count;
DWORD start;

if (argc < 3)
    return 1;

// init threads
printf("Creating threads...");
active = false;
SetThreadPriority(GetCurrentThread(), THREAD_PRIORITY_TIME_CRITICAL);
numthreads = atoi(argv[1]);
count = atoi(argv[2]);
count *= 100000/numthreads;
tid = new HANDLE[numthreads];
printf("%d, count to %Lu\n", numthreads, count);
for (i=0; i<numthreads; i++)
    {
    tid[i] = CreateThread(NULL, 0, thread_func, &count, CREATE_SUSPENDED,
NULL);
    if (tid[i] == NULL)
        {
        printf ("error create thread\n");
        }
    SetThreadPriority(tid[i], THREAD_PRIORITY_IDLE);
    }

// start!
printf("Resuming...\n", MAXIMUM_WAIT_OBJECTS);
for (i=0; i<numthreads; i++)
    ResumeThread(tid[i]);
active = true;

// wait for threads to terminate
start = GetTickCount();
for (i=0; i<numthreads; i+=j)
    {
    j = numthreads-i;
    if (j > MAXIMUM_WAIT_OBJECTS)
        j = MAXIMUM_WAIT_OBJECTS;
    if (WaitForMultipleObjects(j, tid+i, TRUE, INFINITE) == WAIT_FAILED)
        printf("wait failed: error=%u\n", GetLastError());
    }
printf("Time=%dms\n", GetTickCount()-start);

delete[] tid;
}

Thanks
Another myth clarified...

"Ted Tripp" <ttripp@skypoint.com> wrote in message
news:3e70406f@newsgroups.borland.com...
> Thanks for the info folks.  Yes, it is true I do have plenty of looping.
> This is a desktop (so far) app that does multiple http(Indy 9) reads
> simultaneously.  It seem to rock the first 10-20 seconds then slows
down--I
> can see the CPU avail disappear.  I would have thought the reads on my
> clunky DSL would act as a built in sleep (though, of course we can assume
> the customer may have something better than a clunky DSL).  Anyway, I'll
try
> introducing a few Sleeps within the next few days and see if I can get
some
> performance back.  Oddly, I seem to be having more difficulty simply
getting
> the OnWork Bytes/Sec to work properly, but that's a post for a different
ng.
>
> Another question, I vaguely recall reading that threads are initialized
with
> a certain (default?) amount of memory.  I seem to recall something like
> 1MB--maybe this was a nonwindows OS...  Anyway, if true, can this, should
> this, be tweaked?  If not true, NM.
>
> Again, thanks for all the info...
>
> ttripp
>
>

On Thu, 13 Mar 2003 02:27:02 -0600, "Ted Tripp" <ttripp@skypoint.com>
wrote:

>Another question, I vaguely recall reading that threads are initialized with
>a certain (default?) amount of memory.  I seem to recall something like
>1MB--maybe this was a nonwindows OS...  Anyway, if true, can this, should
>this, be tweaked?  If not true, NM.
>

That is the stack size.  Reserving a stack requires part of your
virtual address space (1MB is not necessarily committed, there is a
guard page on the stack that will trigger the stack growth).  Since
the user virtual address space is limited (2GB on most 32-bit versions
of Windows, 3GB on some, not a problem on 64-bit Windows) this places
an upper limit on the number of threads you can have.  You can reduce
the amount of stack a particular thread gets, but approaching 2000
threads is not a good thing anyway.  

Even 100 is a large number of threads.  If your threads are often
blocked waiting for I/O it doesn't hurt to have quite a few to
simplify the sturcture of your code.  If all of the hundreds of
threads are CPU bound, some percentage of the available CPU time will
be wasted with the administrative overhead of switching between the
threads.  But considering how fast the CPU is compared to everything
else in a system including the memory...

Chris Hill
Chris244@aol.com

On Fri, 14 Mar 2003 04:14:11 GMT, Chris244@aol.com (Chris Hill) wrote:

>Even 100 is a large number of threads.  If your threads are often
>blocked waiting for I/O it doesn't hurt to have quite a few to
>simplify the sturcture of your code.  If all of the hundreds of
>threads are CPU bound, some percentage of the available CPU time will
>be wasted with the administrative overhead of switching between the
>threads.

I haven't delved into how Win32 handles this(and maybe 9x is different
than NT flavors) but I know in the old days when OS/2 2.x worried about
being run on 386 machines they "cheated" by just switching the value
in the TSS(this is just from memory so I may get the tasking mechanism
wrong) instead of reloading all the valid info to try to cut down on the
switching overhead.  As I remember there were plenty of threads running,
even when not all that much was going on.  What killed it was when some
communications sw tried to bump up priority to hog the CPU.

I would think with Pentium+ machines it wouldn't be all that much
of an issue.  Processes might be another story where you really
have to save some state.

-- 

"Only choice is an oxymoron."

Well, after some playing around I added a user defined throttle that can be
set and puts a sleep of 20--200 ms within a busy loop.  Also, letting user
define the number of threads--max 100--since I have no idea what kind of
system or bandwidth they will have.  I ran it succesfully at 100
threads--sure, it ran at half speed and eventually got buffer overflows from
my DSL router, but she worked!  Thanks for the input all.  Your commission
checks are in the mail.  There is also good money to be made a well written
thread manual...

ttripp

"Ted Tripp" <ttripp@skypoint.com> wrote in message
news:3e70406f@newsgroups.borland.com...
> Thanks for the info folks.  Yes, it is true I do have plenty of looping.
> This is a desktop (so far) app that does multiple http(Indy 9) reads
> simultaneously.  It seem to rock the first 10-20 seconds then slows
down--I
> can see the CPU avail disappear.  I would have thought the reads on my
> clunky DSL would act as a built in sleep (though, of course we can assume
> the customer may have something better than a clunky DSL).  Anyway, I'll
try
> introducing a few Sleeps within the next few days and see if I can get
some
> performance back.  Oddly, I seem to be having more difficulty simply
getting
> the OnWork Bytes/Sec to work properly, but that's a post for a different
ng.
>
> Another question, I vaguely recall reading that threads are initialized
with
> a certain (default?) amount of memory.  I seem to recall something like
> 1MB--maybe this was a nonwindows OS...  Anyway, if true, can this, should
> this, be tweaked?  If not true, NM.
>
> Again, thanks for all the info...
>
> ttripp
>
>

This is interesting, I am thinking you application, Remy, is interrupt 
driven rather than poll driven, thus no matter how many threads
there is, the only time there is activity is when the device requires
servicing?? And that you are not actually polling and requesting 
servicing of the device?

I think people have the wrong impression about threads, and how they
can be used.
 

"Remy Lebeau (TeamB)" wrote:
> 
> "Dennis Jones" <djones@nospam.com> wrote in message
> news:3e6fd97b$1@newsgroups.borland.com...
> 
> > Are you saying that the program experiences no degredation
> > in performance due to those 100's of context switches, or that
> > the program is not significantly impaired by 100's of context
> > switches?
> 
> Both, actually.
>

"Colin Maharaj" <cbm@cablenett.net> wrote in message
news:3E7DB48E.4C3A39E3@cablenett.net...
> This is interesting, I am thinking you application, Remy,
> is interrupt driven rather than poll driven, thus no matter
> how many threads there is, the only time there is activity
> is when the device requires servicing??

Very possibly.  I'm not privvy myself to how the lowest level code of the
software is written.  It does use APIs and such for manufacturer drivers for
the hardware we communicate with, so I suppose its possible.  There are some
internal helper threads that are polling other threads for status info and
such, however.


Gambit