🚀 go-pugleaf

>I have some questionnaire data from which I wish to derive a "knowledge
>index"   by taking the mean response to several of the questions. 
>However, these are true-false questions and more of the questions are
>true than false.  Moreover, many people are clearly disposed to choose
>True as a response rather than False.  Of course, to the degree that
>they have this bias their knowledge index is inflated.

>I have solved this problem by taking the (unweighted) mean of the mean
>answer to false questions plus the mean answer to true questions.

An alternative is an approach often used for rescaling multiple choice 
exams:
In this case there is a 50% probability of guessing a correct answer.
So all of the wrong answers must be about half of the guesses - the other 
half of the guesses were correct by pure chance.  After calculating the 
probable number of guesses you can subtract this number from the total 
number of questions.  In this case it reduces to, for each person:
                 SCORE = N(correct) - N(wrong)

Of course this does not get you arround the systematic bias created by the 
propensity to answer TRUE more often.
  - - - - - - - - -
 |      _--_|\     |   Hume Winzar
 |     /      \    |   Commerce School,
 |     *_.--._/    |   Murdoch University,
 |           v     |   Perth, Western Australia
  - - - - - - - - -  
                       E_Mail winzar@csuvax1.csu.murdoch.edu.au
                       Phone: (09) 310 7389
                       Fax:   (09) 310 5004

In article <winzar.73.0@newsman> winzar@newsman (Hume Winzar) writes:
>Of course this does not get you arround the systematic bias created by the 
>propensity to answer TRUE more often.

That bias results from a mis-designed questionaire.  If the correct choice 
is randomly chosen (with equal probabilities) and the question worded to 
fit, then any system of picking answers that doesn't depend on knowledge 
about the statement will have equal chances of picking the right and wrong 
answers.  You should also try to make knowledge of the subject matter 
important, e.g. don't do this:

  As everyone knows, 2+2 = 5.    True False

  2+2=4.                         True False

  As everyone knows, unbiased estimates of the variance are essential.
			         True False

Most sponsored surveys are just about this blatant in wording their 
questions to elicit the "correct" answer.  

Duncan Murdoch
dmurdoch@mast.queensu.ca