CONSUMER'S GUIDE: HOW TO EVALUATE METHODOLOGY

OF THERAPY STUDIES

When comparing the differential effectiveness of therapies (Rx)

the following points are relevant:

1. Mini-Experiment: Comparison must assure that any observed Rx

changes are due to the active ingredients of the intervention.

Therefore we must be able to rule out alternative explanations of

the observed differences and/or changes. This process, of being

able to rule out rival alternative hypotheses, is referred to as

establishing the "internal validity" of the study.

2. Intentional (i.e., designed to be effective) factors: Identify the factors

can cause "change".

A. Hopefully the specific effective intervention

B. Providing correct (effective) dosage

C. Having control over active ingredients

D. Trained therapists

3. What extraneous (i.e., nonplanned for) factors cause change?

A. Subject selection biases

B. Nonrandom assignment to treatments (Note: very important!)

C. "Maturation" effects

D. Extra (collateral) therapy input (e.g., readings, inspirational groups)

E. Regression to the mean effects (Law of Large Numbers)

F. Practice effects from test taking

G. Failure to assign therapists randomly to therapies

H. Nonspecific effects

4. What should a good study include?

A. Control group of some kind (Note: issue of ethics here)

B. Available treatment manual (what was actually done!)

C. Subjects representative of some defined population

D. Independent evidence that Rx manual was followed

E. Nonreactive pre- and post-measures indicating change

F. Follow-up; are the effects long lasting?

G. Adequate sample size

H. Clearly stated hypotheses

I. Corrections for making many statistical comparisons within

the same study

5. Case studies, per se, cannot provide evidence that a therapy

is better than no therapy.