diary of a doctoral student, psychometrics, statistics

Dissertation topic: Constructing predictive indexes

The actual working title of my dissertation is: Modeling Social Participation as Predictive of Life Satisfaction and Social Connectedness: Scale or Index?

When I tell people my topic, I usually start with the domain area: social participation as related to life satisfaction in older U.S. adults (my data set is people age 65 and over from the Health and Retirement Study), but really, the topic is a statistical and measurement one. Participation happens to be something I’m personally interested in and fits the statistical problem area, but I could do this same project in a variety of domains with a range of constructs. Maybe I ought to change my elevator speech to start with the statistical/measurement part.

Most psychometrics concerns itself with the measurement of latent psychological constructs like attitudes, intelligence, academic achievement and so forth. Psychometricians have developed sophisticated means of constructing instruments (surveys or assessments, for example) that can measure these latent constructs. The approach taken is often based on either classical test theory or item response theory. Either way, the assumption is that observed data (such as a student’s answers to test questions or a subject’s survey responses) are caused by whatever unobserved trait is intended to be measured.

However, there are some things we want to measure that don’t fit this model. Social participation is one of them. Participation instruments generally ask the respondent to report his or her level of participation in various activities. In a latent factor setting, you would then assume some underlying level of participation that gave rise to the observed frequencies of participation. That’s not quite right though. If someone increases their participation in some area — say by joining an investment club — their overall level of participation goes up. The increase in participation in the investment club seems causally prior to the increase in overall participation. This is the opposite direction of causality than that proposed by traditional psychometric models.

Some people call a measurement instrument developed by some sort of summation of disparate items an index rather than a scale, where a scale follows the latent factor model. The development of such indexes follows a so-called formative measurement model, where what you’re trying to measure is formed of what you observe, in contrast to the development of scales that follows a reflective measurement model, where what you observe reflects the underlying latent factor of interest. In the diagram, the first figure represents formative measurement (observed indicators x1-x3 cause the latent construct eta 1) and the second figure represents reflective (observed indicators y1 to y3 reflect the level of the latent construct).

There has been plenty of criticism of formative measurement, but I think it can be made useful, and that’s the aim of my dissertation project. I’m now at the analysis stage and just beginning to really understand the usefulness and potential of formative indexes.

As an aside, I don’t like to call formative measurement “measurement.” I prefer to think of it as “modeling.” I think what you’re doing with index development is constructing a one- or few-number summary of a lot of individual data items in a way that predicts outcomes of interest. Think of the Apgar score as a good example. It gives you a one number summary of the health of the baby and its likelihood to survive and thrive, but you’re not measuring one thing in particular about the baby. Well, maybe you are measuring overall health. Hmmmm.

To be continued…


2 thoughts on “Dissertation topic: Constructing predictive indexes

  1. I still don’t think you have your direction of causation correct in regards to formative measurement. You say:

    “In a latent factor setting, you would then assume some underlying level of participation that gave rise to the observed frequencies of participation. That’s not quite right though. If someone increases their participation in some area — say by joining an investment club — their overall level of participation goes up. ”

    That’s not quite right either – the latent factor would be some psychological trait – e.g. belongingness, desire to participate – which in turn caused both of those participation rates to increase. You can’t conceptualize that form of participation as causing the other – rather, they are alternate indicators of the same underlying phenomenon (or are possibly at different levels of measurement).

    Your observation about Agpar, as far as I can tell not being familiar with it specifically, is right on – you are collecting disparate specific bits of information and ultimately measuring overall health (the latent trait). The latent trait cannot be measured directly, but the operationalizations that reflect it can. The complexity really comes in when the latent trait is multidimensional, and it is that complexity level that lead many to flee to the relative (although illogical) simplicity of formative measurement.

    1. I agree, what you’re modeling with the reflective model of participation is a drive to participate, not actual participation. That is a problem with using reflective measurement when a formative model is called for — you’re not actually capturing the same thing. A drive to participate is prior to actual participation (what you’re trying to quantify) and increased belongingness may be a result of it.

      Health does seem multidimensional — you have respiratory health, brain health, digestive health. Reflective models handle multidimensionality quite well but if you were to use Apgar’s indicators to model health reflectively you’d likely find multiple latent factors and you wouldn’t come up with a one-number summary. You could use a second-order factor model, but the real purpose is to quantify in terms of predicted outcomes… how well will this baby survive and thrive? Introducing predicted outcomes into a formative model actually expresses the situation of interest. Besides that, there is not necessarily a higher order factor of health even though it’s useful for us to conceive of one. A baby might have had an insult to one system but everything else is fine.

      Formative modeling is not illogical just like multiple regression is not illogical — it is merely a way of expressing that sometimes multiple things combine together to predict other things. I don’t necessarily take a composite variable as something real though. Formative modeling is pragmatic, whereas reflective models often assume a realist epistemology in which latent factors have objective, independent existence.

      Thanks for your comment, and for your blog post summarizing Edwards’ critiques of formative measurement. I know I have a lot more to think about here and feedback is helpful. Meanwhile, I find Bollen’s writings on formative measurement have some good answers to Edwards’ critiques.

Comments are closed.