Extraneous Factors

We use experimental and quasi-experimental designs to insure the internal validity and external validity, or generalizability, of an experiment. Even the hallowed Randomized Control Trial can be compromised if any of the following effect the control and study group(s) differentially. The most common problems with internal validity are 1 :

• History - events that might influence or bias the outcome (E.g., you are studying a behavior modification program for diabetics, and JAMA reports a cure for diabetes that is reported on television.)
• Maturation - changes in the participant (getting older, tired, hungry, etc.).
• Testing - the effect of one test on a second test (For example, if you do a glucose tolerance test before starting the behavior modification program, the fact that they have been tested may cause them to become more cooperative.)
• Instrumentation - Systematic changes in the accuracy of a test may produce a spurious effect.
• Statistical regression - can occur where subjects are selected on the basis of extreme scores (Take this one on faith. Extreme scores tend to move towards the average on a second testing occasion without anything being done to the subjects in the meantime.)
• Experimental mortality - as when subjects drop out of the experiment.
• Selection-maturation interaction (and other interactions) - the same maturation factor may effect different persons differently (e.g., time since the last meal tends to affect diabetic patients more than non-diabetic persons).

Extraneous factors can also jeopardize the external validity, or generalizability, of your experiment. Some of the more common examples are:

• The reactive or interactive effects of testing - where you are trying to generalize to a population that will not have the same test. (E.g., you administer a questionnaire prior to your experiment, and the subjects become sensitized and more cooperative than the general population would be.)
• The interaction of selection and the experimental variable - where your groups are not equivalent, either due to inadequate randomization or to subjects dropping out.
• Reactive effects of experimental arrangements - where the experimental conditions are significantly different from the "natural" conditions.
• Multiple-treatment interference - which can occur, for example, where subjects serve as their own controls, or when multiple treatments are tried on the same patients. (I.e., multiple treatments usually tend not to occur in the "real world", at least not in the same way.)

Internal validity is, quite simply, that your experimental treatment, and only the experimental treatment, was responsible for the outcome.

External validity refers to the degree to which your findings may be applicable in everyday use, outside of the experimental context.

The Randomized Control Trial establishes equivalence of the experimental and control group(s) by creating an equal chance for each participant to be assigned to experimental or control group. This eliminates the problem of differential selection, which can be a problem, for example, when existing groups are used for experimental and control (such groups may be systematically different in ways that would affect the outcome of the experiment).

Campbell, D.T. and Stanley, J.C. (1966). Experimental and Quasi-experimental Designs for Research. Chicago: Rand McNally.