Green, Ha, & Bullock (2010) critique mediation analyses in social science research:
Given the strong requirements in terms of model specification and measurement, the enterprise of “opening the black box” or “exploring causal pathways” using endogenous mediators is largely a rhetorical exercise.
But what is social science anyway? To what extent can we find the “truth” about complex social systems that involve agents with free will and myriad complex, interlinked influences on them?
Perhaps social science is just rhetoric of an advanced sort, carefully constructed arguments based on theory, prior research, data analysis and hunches that describe how the world might work. Over time, some of these arguments are shown to be false, so (ideally) we fix the story up and make it better fit what we’ve observed and what we can deduce from the build-up of evidence and argument so far.
An example from the psychology of narrative identity processing
Pals’ (2006) study of narrative identity processing and adult development is an example of mediation analysis as advanced rhetoric. Here’s the abstract:
Difficult life experiences in adulthood constitute a challenge to the narrative construction of identity. Individual differences in how adults respond to this challenge were conceptualized in terms of two dimensions of narrative identity processing: exploratory narrative processing and coherent positive resolution. These dimensions, coded from narratives of difficult experiences reported by the women of the Mills Longitudinal Study (Helson, 1967) at age 52, were expected to be related to personality traits and to have implications for pathways of personality development and physical health. First, the exploratory narrative processing of difficult experiences mediated the relationship between the trait of coping openness in young adulthood (age 21) and the outcome of maturity in late midlife (age 61). Second, coherent positive resolution predicted increasing ego-resiliency between young adulthood and midlife (age 52), and this pattern of increasing ego-resiliency, in turn, mediated the relationship between coherent positive resolution and life satisfaction in late midlife. Finally, the integration of exploratory narrative processing and coherent positive resolution predicted positive self-transformation within narratives of difficult experiences. In turn, positive self-transformation uniquely predicted optimal development (composite of maturity and life satisfaction) and physical health.
This study was correlational, so that’s the first reason that strict causalists would dispose of it. It also studied mediation, so even if it were some sort of randomized experiment, there would be questions about its suggestions of causality. But the researcher doesn’t just run the mediational analysis and then declare that she’s shown what she wanted to show. She places the correlational findings in the context of theory and makes an overall argument for her hypothesis while noting the limitations of the approach:
A second limitation of this study is that although the hypotheses reflect theoretically driven ideas about cause-effect relations (e.g., coping openness stimulates exploratory narrative processing; coherent positive resolution leads to increased ego-resiliency), the correlational design did not allow for analyses that would support conclusive statements regarding causality. The longitudinal findings were consistent with causal patterns unfolding over time but did not prove them. Thus, an important direction for future research on narrative identity processing will be to examine its causal impact, ideally through studies that closely examine the connection between changes in narrative identity and changes in relevant outcomes. In one recent study, for example, individuals who wrote about a traumatic experience for several days displayed an increase in self-reported personal growth and self-acceptance, whereas those who wrote about trivial topics did not show this pattern of positive self-transformation (Hemenover, 2003). This finding supports the idea that when people fully engage in the narrative processing of a difficult experience, their understanding of themselves and their lives may transform in ways that will make them more mature, resilient, and satisfied with their lives. Findings such as these reflect the growing view that the narrative interpretation of past experiences—the cornerstone of narrative identity—constitutes one way adults may intentionally guide development and bring about change in their lives (Bauer et al., 2005).
Is this research useful even if causality and mediation has not been proven? I think it is. We don’t know for sure which way causality runs among the various traits and behaviors studied (it probably runs in multiple directions) but Pals makes a good argument that someone with coping openness may engage in exploratory narrative processing of difficult life events and this, in turn, may drive a maturing process. In the second mediational hypothesis, she argues that developing coherent positive resolutions in that narrative processing of life events might lead to increased ego-resiliency. Are these analyses and arguments of practical use? I think yes.
The remembering self creates stories and needs those stories to make sense of experience. Research like Pals (2006) gives insight into what sort of stories might be most useful in leading towards maturity and psychological resiliency:
- Development of the stories should use an open and exploratory style rather than closed and defensive.
- The ending of the story should reflect some sort of positive resolution.
So mediation analysis, even of the non-experimental sort, can be useful. Okay so maybe it’s not like the scientific finding that lack of Vitamin C causes scurvy, but that doesn’t make it useless or unscientific.
Philosophers of science would have something more sophisticated to say about this. My point is that science doesn’t happen exactly according to the “scientific method” you learned in high school. In many ways it is just advanced rhetoric that’s (ideally) grounded in careful analysis, thoughtful theorizing, and an understanding of prior research.
Green, D. P., Ha, S. E. and Bullock, J.G. (2009) Enough Already About “Black Box” Experiments: Studying Mediation is More Difficult than Most Scholars Suppose. Annals of the American Academy of Political and Social Science 628, 200-08. Available at SSRN: http://ssrn.com/abstract=1544416
Pals, J.L. (2006). Narrative identity processing of difficult life experiences: Pathways of personality development and positive self-transformation in adulthood. Journal of Personality 74(4).