🚀 go-pugleaf

< tom.mead@dartmouth.edu >
Thomas L. Mead (Thomas.L.Mead@dartmouth.edu) wrote:

: I've done some work with medical meta-analysis ("M-A") studies, where
: my job is to perform the literature search (usually rather vast &
: comprehensive & intentionally overbroad).   Then I do some
: post-processing of the results, such that references are in a database
: program.  I also get involved with filtering these results (determining
: which studies gotten from the literature search are likely to be
: subjected to M-A techniques...)

: Sometimes, when I read the METHODS sections of published meta-analysis
: research (in any field), I'm disappointed by what I consider to be
: cursory or inadequate descriptions of the literature search which
: precedes the statistical work.   ("We searched MEDLINE..." just doesn't
: say nearly enough to me!)

Meta-analysis in the area of psychiatric treatment seems to suffer 
from two problems  *besides*  what you mention, both of which make the
outcomes ABSURD, rather than just incomplete.

 -  The Treatments in the study do not match the stated intent.  
 -  The populations under study differ vastly, but that is not even
          mentioned.
 -  The outcomes are not firmly defined.

The only decent medical meta-analyses that I think of, off-hand, 
are the ones that follow the prescription of using SIMPLE 
treatment definitions, and SIMPLE outcome (namely, Death within
a stated interval).

Oh, and the performance of various meta-analyses is often carried 
out without confirming that the results on hand are homogenous 
in a  *statistical*  sense.  That is, the SAME treatment will not 
legitimately show a 1 S.D.  effect in BOTH directions, in two 
different studies that legitimately can be combined for an average.

(I don't know if this helps you directly, but there is a message
here, that at least a few of us have a lot of *scorn*  for what
some meta-analyses have pretended to show us...   just another
inept literature review, but decorated with numbers).



Rich Ulrich, biostatistician              wpilib+@pitt.edu
Western Psychiatric Inst. and Clinic   Univ. of Pittsburgh