research

Experimental and quasi-experimental research designs

Ph.D. Topics : Research and Evaluation Methods

In an experimental design, subjects are randomly assigned to groups for different levels of treatment (or no treatment, i.e., the control group). In a quasi-experimental design, subjects are not randomly assigned to treatment; there is no randomization.

Random assignments of subjects helps control for participant differences, one of the main sources of threats to internal validity of a research study. Random assignment of subjects doesn’t guarantee that there are no participant differences; especially with smaller sample sizes, you may need to take steps to control for participant differences even after randomly assigning them to treatment levels. For example, administering a pre-test will control for different levels of ability or achievement prior to intervention. Measuring moderators such as demographics (gender, age, race, socioeconomic status) and including those in your analysis may help further isolate causal relationships between interventions and outcome. In some experimental designs, researchers may match participants across control and treatment so that each pair of participants can be treated as one virtual participant (Gliner & Morgan, 2000), giving a pseudo-within-subjects design.

A randomized experimental design with pre-test and post-test controls for threats to internal validity from participant characteristics but leaves some threats uncontrolled, specifically testing effects and bias from selective attrition. Testing effects–for example the possibility that taking a pre-test will help both control and treatment group participants do better on the post-test thus obscuring the actual treatment effect–can be controlled by a Solomon four-group design (Gliner & Morgan, 2000). In this design, there are two control groups and two treatment groups (assuming just one level of intervention). One control group and one treatment group takes a pre-test; the other control and treatment groups do not. This allows the potential testing effect to be teased out.

What if you can’t randomly assign subjects to treatments? This is a common problem in educational and other social settings. For example, if you are testing the introduction of a new curriculum it is unlikely you can randomly assign students to that curriculum. Students come to you in intact groups. In this case, you may choose to use a quasi-experimental design in which treatments are assigned to groups. The treatments may be assigned randomly (e.g., pick classrooms out of a hat to decide which intervention they will get) or purposively (e.g., principals at a school may have some say over which classrooms get an untested vs. the standard curriculum).

Like experimental designs, quasi-experimental designs may be improved by the use of a control group, measuring moderators and incorporating them into the analysis, or matching participants on factors that relate to the measured outcomes (Gliner & Morgan, 2000).

One type of quasi-experimental research design is the time series design, in which many observations are made over time, both without intervention and with intervention (Gliner & Morgan, 2000). Multiple observations are used to establish a baseline that shows an (ideally stable) level of the outcome of interest over time. Then multiple observations are made during intervention, ideally showing a change due to intervention. Then the treatment may be withdrawn, again in an attempt to isolate the relationship of treatment to observed outcome. This may be used with or without a control group. A single-subject design is a common time-series design, in which one or a very few subjects are followed through one or more baseline and treatment phases.

References

Gliner, J. A., & Morgan, G. A. (2000). Research Methods in Applied Settings: An Integrated Approach to Design and Analysis. Mahwah, N.J: Lawrence Erlbaum.

Advertisements