🚀 go-pugleaf

I have submitted this to the CUGDUS Software Library. Here is the README file.


/******************************************************************************

 ####                           
##  ##                          
##  ##  ##  ##   ####   #####   
##  ##  ####### ##  ##  ##  ##  
## ###  ####### ##  ##  ##  ##          An NQS queue manager
 ####   ## # ## ##  ##  ##  ##  
   ###  ##   ##  ####   ##  ##  

	email: simonm@mel.dit.csiro.au
******************************************************************************/

QMON is a queue monitor program that holds and releases NQS jobs in a
"fair" manner, stopping users from "stuffing" queues with many consecutive
job submissions.

Scenario: User "hogg" decides that he wants to submit 20 jobs to the queue
cray.average . The 20 qsub's go into the queue cray.average, and his first
job runs for 1 hour. Then the next job in the queue starts to run (hogg's
second job), hoggs' 20 jobs will be run consecutively, being unfair to users
who have only 1 or 2 jobs waiting in the queue. This is what happens unless
an operator with queue privileges steps in and starts holding and releasing
jobs manually.


On the unicos-l@violet.berkeley.edu mailing list, a suggestion was made on
how to overcome this problem:

-------------
Date: Tue, 24 Mar 92 14:56:51 CST
From: rbrown@ncsa.uiuc.edu (Rich Brown Cray Operations)
Message-Id: <9203242056.AA23280@mars.ncsa.uiuc.edu>
To: unicos-l@violet.berkeley.edu
Subject: limiting number of queued requests in NQS

What we did was to rename /usr/bin/qsub to something else ('quesubmit', below)
and replace qsub with the following script:

#!/bin/sh
if [ `qstat -a -u $LOGNAME 2>/dev/null | grep -c $LOGNAME` -lt 10 ]
then
        exec /usr/bin/quesubmit "$@"
else
        echo "NCSA policy does not allow more than 10 "
        echo "jobs queued, simultaneously, per user."
fi
################

It would probably be cleaner to make a mod to the nqsdaemon, but this works for
us.
-------------


But this solution still lets a user effectively "stuff" a queue.


QMON will run as a continuous process using the UNICOS sleep command to wake
up every few minutes and check the state of the queues it is monitoring,
OR qmon will run from an NQS submission, resubmitting itself every few minutes.

QMON needs to be run by a user with queue operator privileges, and uses the
UNICOS qmgr program to hold and release jobs.

If a user is running a job in any particular queue, qmon will hold all other
jobs in that queue owned by the user, allowing everyone elses jobs to be
placed in the queue ahead of that user.

When a user's job finishes, qmon will release that user's held jobs, but not
before someone elses job starts running.

QMON does not require any interaction. If a queue operator wants to hold
certain jobs, release jobs that have been held by qmon, qmon will accept it
and not override external changes in the queues.

If qmon does fail for some reason, or qmon is prevented from executing, a
script is supplied that will "undo" i.e. release jobs that were held by qmon.




QMON is (hopefully) easy to install. It is a matter of:
* running a configuration script that asks questions about the installation
	(comes with default values)
* make an account with operator privileges, or use the operator account to
	compile and run qmon.

QMON is written in C
The configuration program and external programs use /bin/sh (Bourne shell)
The qmon NQS resubmitting job uses /bin/ksh (Korn shell)

QMON was developed and tested on a Cray Y-MP/216 running UNICOS 6.1.5
and a Cray Y-MP4E/364 running UNICOS 6.1.6


--
Simon.McClenahan@mel.dit.csiro.au  \_\_ CSIRO Supercomputing Support Group
CSIRO Division of Information Technology _\_\_\_\_\_\_ tel: +61 3 282 2623
723 Swanston St, Carlton 3053 AUSTRALIA \_\_\_\_\_\_\_ fax: +61 3 282 2600
       The four major food groups:  Fresh, Fast, Frozen, and Spoiled.