🚀 go-pugleaf

In article <3128ff8b.666031216@news.clark.net>, anderson@clark.net (Thom Anderson) says:
>
>I am starting to port a design that was meant to be implemented in Ada to a
>C++/UNIX platform.  I don't want to get involved in any language religion here,
>but it will be much harder to handle tasking with C++.  My best alternative
>(maybe only - since I must be POSIX compliant) is to use pthreads.
>
>I have used pthreads from C before but not from C++.  I have a couple of ideas
>in mind but neither is elegant.  Does anyone have any experience with
>C++/Pthreads and no of some good design solutions, tips, or ideas???
>

I have implemented a C++ Base Thread class that allows threads cross 
communications using message queues. 

The idea is that you have a base thread class, with public interface
access functions to create a thread (can be implemented using Pthread
or whatever thread package you have), and to READ/WRITE from thread
Input and Output Queues. You also need to have a pure virtual 
function that is the address of thread function code itself which the
derived classes have to provide. This way each specialized class can use
different thread functions. 

Now, assume you have created 2 threads objects, A and B. A uses function
A_foo() as the thread function, and B say uses B_foo() as the B thread
function.

For A to send a message to B, A will invoke a public access function in B
that will deposit a message inside a private Queue in B and will then wake
up B (B thread id is saved in private part of B class object )and return. 

Each thread can once in a while can check its input Queue to see if it has 
messages to process. If no message, it will sleep until awakened (or 
sleep for certain period, then tries again etc...).

If you want to implement a driver type of thread, (passive thread) you can
also have the thread object contain an output Queue (in addition to 
Input Queue), and the thread will deposit processed messages (work 
completed) in its output Queue and will then wake any waiting thread on
its output Queue (the readers). (in first-in-first_out fashion).

Each reader will invoke a READ access public function in the thread class,
where this READ access function will access the thread private output
Queue and if there is data, it will remove it, and return, else it will
sleep (block) on the Queue (saves its thread_id in waiters Queue) so
that when the driver thread has data to send out, it will put it in its
output Queue, check the waiters/readers queue, wake up the first in 
line using the reader thread_id). Offcourse you do not have to have
the read block if there is no data. you can have a time_out value, or
other combination.

I do not know Pthreads much, so I do not know if you also need some 
kind of signal number to use in addition to the thread_id to wake up
a thread (You need to look at the Pthread Wake thread API), but if you
do, that is easy to also pass along with the thread_id to allow the 
other thread to wake your thread. (i.e. each Input and Output Queue 
for each thread  will have a signal number associated with it).

All in All, you need to spend good time designing the thread class,
and hide all the threads-cross communications in the private part of
the class. Using Queues to accomplish the message exchanging is the 
most natural way.

One thing to worry about is how much time a caller thread will spend
in the access function of the other thread object. While executing 
the access function inside one thread object the client thread is 
executing in its own  context ,which might have higher priority that
the called thread object, and so things like blocking inside the 
access function need to be thought out well to make sure that is 
what you want to do. (to prevent among other things possible dead-lock).

It is really not that hard, I have implemented the complete class in about
1-2 week, it is 3 layers in class hierarchy, and it is about 500-600 
lines of code (and this was the first time I looked at this problem
of using C++ classes and threads.

offcourse you need to have access to thread-safe Queue classes.

If you on the other hand decide to use Ada , you'll have easier time,
since Ada as you know already solves many of these problems for you, 
by providing tasks and protected objects, the Requeue facility, Rendezvous,
Asynchronous transfer of control, and many other features for
task-to-task interactions.

Nasser

Thom Anderson wrote:

> I have used pthreads from C before but not from C++.  I have a couple of ideas
> in mind but neither is elegant.  Does anyone have any experience with
> C++/Pthreads and no of some good design solutions, tips, or ideas???

One approach that works just fine:  use it from C++ like you use it from C.

If your C++ environment has good pthreads supports, you'll be able to forget
about pushing/popping cleanup handlers since the C++ destructors will do all
of that stuff for you when your thread is canceled.  (Cancel != Kill ...)
You _really_ want that support, else you won't be able to support cancellation
without lots of additional effort.

One of the really nice techniques is to have a "Locker" class to grab mutexes
as needed, and then release it automatically on all exits.  Something like:

	class Locker {
	  public:
				Locker (pthread_mutex_t *lock)
				: mutex (lock) { pthread_mutex_lock (mutex); }
				~Locker () { pthread_mutex_unlock (mutex); }
	  
	    int			wait (pthread_cond_t *cv)
				{ return pthread_cond_wait (cv, mutex); };
	
	  private:
	    pthread_mutex_t	*mutex;
	};

and then use it like

	SomeClass::member_function ( ... arguments ...)
	{
		... do a bunch of work

		Locker (&some_mutex);

		... do more work
		... The mutex is held during this work, and always
		... released on all exits from this function
	}

That kind of class really helps get rid of the bugs you have due
to locks not getting released uniformly on all codepaths.

I've never found a "thread" class to be of any help except as an
excercise to show basic C++ and POSIX.1c skills.  CFRONT used to
have a thread class used for simulations, and there are folk who
evidently like such classes.  Having programmed MT/C++ software
for the past four years or so, I can testify that they're not
necessary.  But of course, your preferred style can differ.

- Dave

David Brownell wrote:
> One of the really nice techniques is to have a "Locker" class to grab mutexes
> as needed, and then release it automatically on all exits.  [...]
> That kind of class really helps get rid of the bugs you have due
> to locks not getting released uniformly on all codepaths.

Hmmm...this approach makes me nervous.  I assert that it's better to neglect to
unlock a mutex than to unlock it when you shouldn't have.  

For instance, if a thread holding a mutex via the "Locker" class terminates
prematurely (due to an exception or cancellation), then the mutex will be
unlocked automagically, despite the fact that whatever it was supposed to
protect is now likely to be in an inconsistent state.  If the process manages
to survive losing this thread, then any subsequent accesses to the shared data
could easily be disastrous in subtle ways.  It would have been much better to
have just left the mutex locked!  (That is, a deadlock would be much easier to
debug than a memory corruptor.)

The missing piece is that the destructor for Locker, prior to releasing the
mutex, must restore the shared data that the mutex protects to a consistent
state.  In order to do this, the destructor requires intimate knowledge of what
the lock is protecting -- this invalidates the "generic" approach.

It would be much better to encapsulate the mutex together with the shared data
as some sort of compound object.  Then it would be feasible to have the proper
clean-up built in.  And then it would be OK to be unlocking the mutex
automagically.  :-)

-- 
------------------------------------------------------------------------
Webb Scales                                Digital Equipment Corporation
scales@wtfn.enet.dec.com                   110 Spit Brook Rd, ZKO2-3/Q18
Voice: 603.881.2196, FAX: 603.881.0120     Nashua, NH 03062-2711
         Rule #12:  Be joyful -- seek the joy of being alive.
------------------------------------------------------------------------

Webb Scales <scales@zko.dec.com> writes:

> David Brownell wrote:
>> One of the really nice techniques is to have a "Locker" class to grab mutexes
>> as needed, and then release it automatically on all exits.  [...]
>> That kind of class really helps get rid of the bugs you have due
>> to locks not getting released uniformly on all codepaths.

> Hmmm...this approach makes me nervous.  I assert that it's better to
> neglect to unlock a mutex than to unlock it when you shouldn't have.

Since the question was about pthreads in C++, you should use the
technique presented nevertheless.  The problem is that you can leave
the critical section not only in the "normal" way (where unlock() gets
executed), but also by means of an exception thrown deep in your
calling chain.  It's simply too easy to forget an appropriate catch clause.

Harald Fuchs wrote:
> It's simply too easy to forget an appropriate catch clause.

But, that's my point exactly!  If it's easy to forget the catch clause,
then it is even easier to forget to rectify the state of the shared
data.  If you fail to unlock the mutex, your application will soon
deadlock, which will make it obvious that something is wrong and which
will suggest areas to examine; on the other hand, if the mutex is
silently unlocked, then the problem my not surface until another thread
at some distance away in the code attempts to use the shared data, and,
at that point, all indications of the source of the problem are gone.

------------------------------------------------------------------------
Webb Scales                                Digital Equipment Corporation
scales@wtfn.enet.dec.com                   110 Spit Brook Rd, ZKO2-3/Q18
Voice: 603.881.2196, FAX: 603.881.0120     Nashua, NH 03062-2711
         Rule #12:  Be joyful -- seek the joy of being alive.
------------------------------------------------------------------------

Webb Scales wrote:
> 
> Harald Fuchs wrote:
> > It's simply too easy to forget an appropriate catch clause.

... but if you have viable thread support from your C++ system,
that'll never be an issue.  That was a premise in my original
posting, as I recall:  the C++ exception runtime interacts in
the natural way with the POSIX cancellation interface (i.e. the
destructors are the C++ version of the C "cleanup handlers").

Admittedly POSIX.1c doesn't have a C++ binding yet.  But the
rationale makes clear what's intended:  use the language's
native exception handling facilities to handle cancellation.

> But, that's my point exactly!  If it's easy to forget the catch clause,
> then it is even easier to forget to rectify the state of the shared
> data.

These go together in all good MT-aware programming styles, just like
policies like "mutex state isn't changed by function calls".  As with
all style guidelines, they can be broken if you know what you're doing;
I tend to call that "rocket science mode".

- Dave

Webb Scales wrote:
> 
> David Brownell wrote:
> > One of the really nice techniques is to have a "Locker"
> > class to grab mutexes
> > as needed, and then release it automatically on all exits.  [...]
> > That kind of class really helps get rid of the bugs you have due
> > to locks not getting released uniformly on all codepaths.
> 
> Hmmm...this approach makes me nervous.  I assert that it's better
> to neglect to unlock a mutex than to unlock it when you shouldn't have.

Depends on how you're using the mutex; both bugs are deadly.  Did the
victim die because of the loss of blood first, or the kidney failure?
(Don't you just hate it when police beat up suspects?)


> For instance, if a thread holding a mutex via the "Locker" class terminates
> prematurely (due to an exception or cancellation), then the mutex will be
> unlocked automagically, despite the fact that whatever it was supposed to
> protect is now likely to be in an inconsistent state.

Huh???  What kind of buggy algorithm are you assuming?

Cleanup processing for exceptions/cancellation is required to leave
the resources in a consistent state ... else the cleanup processing
is buggy.  The ** whole point ** of the cleanup processing (destructor
handling etc) is to keep state consistent.  Any C++ instructor who is
not making this clear needs to be immediately replaced.

Now ... given that C++ stack cleanup is done on each function call
exit, and hence isn't specific to exceptions/cancellation, I'd claim
that bugs in cleanup with a "Locker" class are found VERY fast.  I
can support that with my own experience, FWIW!


> The missing piece is that the destructor for Locker, prior to releasing the
> mutex, must restore the shared data that the mutex protects to a consistent
> state.  In order to do this, the destructor requires intimate knowledge
> of what the lock is protecting -- this invalidates the "generic" approach.

Well, I never said that you would _only_ have a "Locker" class!  I called
it a _technique_ ... meaning, "modify as appropriate".  You're talking as
if I'd said "here's a complete general purpose library", which I didn't.
I'd claim there is no such critter, in fact.

Typical programming style for mutexes is that functions have an
invariant:  the mutex state doesn't change as a consequence of calling
the function.  The "Locker" class I posted did that just for mutexes,
since that generalizes naturally and is an appropriate starting point
for such discussions.

More complex invariants can be encapsulated in other classes rather
easily, and should be.  As in fact Webb noted later.  I didn't have time
to write out a complete C++/MT style guide; that point is a logical
combination of (a) encapsulation, and (b) the "Locker" technique.

- Dave

David Brownell wrote:
> 
> Webb Scales wrote:
> >
> > David Brownell wrote:
> > > One of the really nice techniques is to have a "Locker"
> > > class to grab mutexes
> > > as needed, and then release it automatically on all exits.  [...]
> > > That kind of class really helps get rid of the bugs you have due
> > > to locks not getting released uniformly on all codepaths.
> >
> > Hmmm...this approach makes me nervous.  I assert that it's better
> > to neglect to unlock a mutex than to unlock it when you shouldn't have.
> 
> Depends on how you're using the mutex; both bugs are deadly.  Did the
> victim die because of the loss of blood first, or the kidney failure?
> (Don't you just hate it when police beat up suspects?)
> 
 [...]
> 
> More complex invariants can be encapsulated in other classes rather
> easily, and should be.  As in fact Webb noted later.  I didn't have time
> to write out a complete C++/MT style guide; that point is a logical
> combination of (a) encapsulation, and (b) the "Locker" technique.

But that's exactly what Webb was trying to say. The solution is, as you
say, a COMBINATION of encapsulation and "the Locker technique". Locker
alone will not solve most problems, and may create more. By claiming
that "it" is a solution, you may mislead some folks to believe that
putting Locker into a routine will solve all possible problems. That's
incorrect and dangerous.

Having classes with destructors that clean up state, as TRIVIALLY
DEMONSTRATED by Locker, is clearly the best way to apply C++ semantics to
the problem of protecting thread state invariants against "unusual"
routine exit. But the true solution in nearly all cases must go beyond
the triviality of Locker to restore data invariants, not merely the state
of the mutex!

> - Dave

------------------------------------------------------------------------
Dave Butenhof                              Digital Equipment Corporation
butenhof@zko.dec.com                       110 Spit Brook Rd, ZKO2-3/Q18
Phone: 603.881.2218, FAX: 603.881.0120     Nashua, NH 03062-2711
                 "Better Living Through Concurrency"
------------------------------------------------------------------------

Dave Butenhof wrote:

> Having classes with destructors that clean up state, as TRIVIALLY
> DEMONSTRATED by Locker, is clearly the best way to apply C++ semantics to
> the problem of protecting thread state invariants against "unusual"
> routine exit. But the true solution in nearly all cases must go beyond
> the triviality of Locker to restore data invariants, not merely the state
> of the mutex!

So I guess we all agree.  The original question was a pretty vague
one, and my response was therefore generic.  When you apply generic
techniques, a "cookbook" approach doesn't work.  I did what I could
to make that clear, but in the discussion all of the original context
was lost.  Part of the peril of being taken out of context!
-- 
David Brownell <brownell@ix.netcom.com>

Webb Scales (scales@zko.dec.com) wrote:
: David Brownell wrote:
: > One of the really nice techniques is to have a "Locker" class to grab mutexes
: > as needed, and then release it automatically on all exits.  [...]
: > That kind of class really helps get rid of the bugs you have due
: > to locks not getting released uniformly on all codepaths.
: 
: For instance, if a thread holding a mutex via the "Locker" class terminates
: prematurely (due to an exception or cancellation), then the mutex will be
: unlocked automagically, despite the fact that whatever it was supposed to
: protect is now likely to be in an inconsistent state.  

Good point.  In my use of Lockers, I always pass a pointer to the surrounding
(monitor) object so that the Locker can find and acquire the local lock, e.g.,

     Locker exclusion(this);

Thus, the Locker's destructor can assert the monitor's invariants or
check them and attempt to tidy up.  What I'd really like is a way for
my destructor to know that it is being called through the processing
of an exception and to have access to that exception object.

Tom Payne (thp@cs.ucr.edu)