🚀 go-pugleaf

Hi,

I wonder if any of you could help me with what might be a trivial
problem and might not be. I am not a mathematician so don't have a clue
where to start.

The problem is this: (apologies if it seems long winded)

I use a non-parametric statistic based on signal detection theory (ROC
analysis) to give a measure of the distinctivness(overlap) of 2
distributions of numbers.
The method is really straightforward and I provide a simple example
below.

distribution 1 (d1):
1 2 4 6 7

distribution 2 (d2):
3 5 8 9 10

for above:
nd1 = 5 (number of observations in d1)
nd2 = 5 (number of observations in d2)
N = nd1 + nd2 = 10 (total number of observations)

In order to generate the measure you step through an increasing
criterion and plot the proportion of d1 which is >= the criterion v the
proportion of d2 which is >= the criterion.
for the above distributions this would be:

crit     p(1)     p(2)
0         1.0      1.0
1         1.0      1.0
2         0.8      1.0
3         0.6      1.0
4         0.6      0.8
5         0.4      0.8
6         0.4      0.6
7         0.2      0.6
8         0.0      0.6
9         0.0      0.4
10       0.0      0.2
11       0.0      0.0

The area under the curve on the plot of p(1) v p(2) on a unit square
gives the measure:
in this case = 0.2.

So the possible range of values is 0.0 - 1.0; a value of 0 or 1
indicates that both distributions are completely unoverlaping wheras a
value of 0.5 indicates that the distributions are identical.

The problem I have is in assessing a significance level for the value I
obtain from the above type analysis given nd1 and nd2 can be of any
length and unequal.

What I currently do is feed the values of d1 and d2 into a computer and
generate a distibution of possible values which could be obtained by
keeping the number of values in each distribution the same but
randomising which distribution any particuar value is assigned to
(repeated about 4000 times). The distribution is then used to give the
confidence limits of whether my actual observed value arose by chance or
not. The generated distribution looks pretty normal with mean 0.5. The
varience decreases with increasing total number of measures (N) and
increases as the tw0distributions get more unequal in length (as
|nd1-nd2| increases)

My question (finally!) is this.
Given that there are really only 2 variables which change: nd1 and nd2
(as the actual numbers are just ranked anyway) Is there a mathematical
solution which could give the distribution of all possible values from
random assignment to d1 or d2 without having to run the simulation
several thousand times. I could then use the curve from this equation to
test for significance.

Many thanks in advance for any help and apologies if this is a
longwinded problem with a trivial solution.


--
==============================================
Jon Dodd
Department of Physiology; University of Oxford
Parks Road; Oxford; OX1 3PT
tel 01865-282483; fax 01865-272469
http://waller.physiol.ox.ac.uk/~jvd
==============================================