🚀 go-pugleaf

Ryan Underwood wrote:
> [This is crossposted to comp.lang.asm.x86 due to the instruction level issue
> below]
> 
> Hi,
> I'm getting really puzzled here.  The basic story is that we have a 4 way Xeon
> machine with hyperthreading enabled (8 logical processors).  The application is
> a fairly straightforward SysV (not POSIX) shared memory application.  The
> shared memory region is declared as a volatile double *.  We are using GCC
> 4.0.2 with -O0 and a 2.6.6 Linux kernel with RTLinux extensions.
> 
> One of the processes sets up essentially a RPC call by placing several double
> arguments in the shared memory region along with a function ID, and then
> setting a flag to notify the other process that a call is waiting.  The other
> process is doing soft-realtime computational work as fast as it can with no
> system calls, and occasionally checks the flag mailbox to see if a RPC is
> waiting.  If it is, then it reads the function ID and arguments and goes to
> work on that task.
> 
> This seemed to work fine in my test cases, but eventually broke down for a
> different user.  The shared memory is written to and verified by the writer
> process, but the computation process _never_ "sees" the flag even if allowed to
> run forever.  I can get the computation process to "see" the flag in several
> ways:
> 
> 1) Insert a sleep of between 1 and 10ms before the flag is polled (less than
> 1ms is not sufficient)
> 
> 2) Print the contents of the mailbox on the computation process side before
> polling the flag
> 
> Running under gdb or valgrind does not affect the bug.
> 
> Based on the above, my hypothesis is that in the cases where the condition
> improves, it is because a system call has been taken, and thus the registers
> clobbered.  In the case where the sleep is insufficient, I think it is because
> short sleep durations are executed as busy waits instead of forfeiting the
> processor.
> 
> However, as a matter of experimentation, I changed the memory access in the
> computation process from:
> 
> 	if (m_Mailbox[STALE_MBOX] != 1)
> 		{ /* these actions are never taken */ }
> 
> which generated the following assembly:
> 
> 	movl    8(%ebp), %eax   # this, this
> 	movl    32(%eax), %eax  # <variable>.m_Mail
> 	addl    $288, %eax      #, D.40882
> 	fldl    (%eax)  #* D.40882
> 	fld1
> 
> to: [with knowledge that sizeof(double)==2*sizeof(uint32_t)]
> 
> 	if (((uint32_t*)m_Mailbox)[STALE_MBOX*2] != 1)
> 		{ /* these actions are taken when mbox value is set to 0 */ }

Why all this casting ?

It'd be interresting to see what differences there were if you made
the flag a simple int, with the volatile keyword too.

(But really - you should be protecting this using semaphores)