Tag Archives: lms

But what about learning? Recognizing the signaling function of higher education

I spent just a couple days at the Learning Analytics and Knowledge 2012 conference this past week in Vancouver. Because of personal stuff going on I couldn’t attend the whole conference, but by watching some live streaming, following the conference twitter channel, reading some live blogs, and browsing some of the papers afterward I put together a decent understanding of the topics covered and the issues that came up. Of course, I had far less time than I wanted to connect with people there, but I have hopes that I can address that over the next year online and then go all out for LAK13 in Belgium.

Some attendees felt a disconnect that many vendors and researchers are looking at retention (students re-enrolling across terms) and course completion rather than learning itself. But aren’t we doing learning analytics, they asked?

At Pearson, I am working on exactly the problem of retention and course success. I work mainly with data from Pearson’s LearningStudio hosted learning management system. I sometimes have access to learning outcomes data but only very rarely; I can’t count on having that.

The data scientist is not drunk

Is my work a a case of the drunk looking for her keys under the streetlight? I don’t think so, and it’s not just do to with the profitability of institutions or their need to demonstrate adequate program completion rates and year-over-year retention to accreditation bodies.

Students enroll in higher education programs for many reasons, not all of which have to do with learning. We would all hope that after two or four or more years taking coursework a student will have better skills and cognitive capacity than when she started. But bachelor’s and associate’s degree programs today require students to take many classes that are not relevant to them or to their future work, that merely serve as hurdles to jump over to acquire the degree. The learning is not enough, because education serves a signaling purpose in addition to working to improve a student’s cognitive capacity.

As people working in education, we have to recognize the reality of higher education today, that it is only partially about learning. I’m talking from a purely U.S.-based perspective, as that’s my focus. In the U.S. today, bachelor’s degrees are a basic entry ticket to a decent job in many cases. In fact, credential inflation means that now a master’s degree is required for entry into many professions.

Some economists think higher ed degrees primarily function as signals to employers, that they are not mainly about increased learning or cognitive capacity. On this theory, a student’s degree shows that he has qualities valued by employers: conformity, conscientiousness, a willingness to defer gratification. If signaling is true in some situations or in some ways, that means it’s not enough for a student to take some classes and merely learn what they need to use on the job. They need to get through an entire program, arbitrary requirements and uninteresting or useless classes included.

There’s good evidence that signaling theories of education do not tell the whole story about higher education, that returns to education do indeed reflect the additional skills that graduates in various disciplines bring to the job market. The signaling theory may be true in part but as we might hope, education is also about actual preparation and learning–I don’t mean to say it is not.

I also don’t mean to reduce higher education to job market preparation, though I do question degrees that don’t represent a sound economic investment. The cost of post-secondary education today is such that we can’t divorce it from its role in linking people with economic opportunity.

Are learning and course completion orthogonal?

One presenter called completion and learning orthogonal outcomes. At least one attendee in that session took issue with that. We would all hope this is not the case — we hope that students don’t successfully complete classes without learning anything — but can’t everyone think of a class they were required to take but didn’t take anything away from it? I just completed a Ph.D. and virtually all of the cognate classes were worthless, annoying and time-consuming efforts that I had to complete if I wanted my degree. There was almost no learning taking place in those courses, and I am a highly motivated and engaged learner, taking courses that I thought would be interesting. Sometimes students do just need to complete a course, learning or no.

Certainly we should work toward making every course a worthwhile learning experience for students but in the real world there are always going to be some classes that aren’t that for one reason or another. Assuming that completing a particular program is a good thing for a particular student (a somewhat questionable assumption in this era of heavy student loan debt and low-value degrees), helping them get through all their courses successfully regardless of learning is a good in itself.


Links for March 30, 2012

The new LMS product: You [Audrey Watters/Hack Education]. On Blackboard’s recent strategy change to embrace open source and acquire MoodleRooms and Netstop. The value is in the data, not in the LMS software.

Are undergraduates actually learning anything? [Richard Arum and Josipa Roksa/The Chronicle of Higher Education. For many students, college doesn't improve their critical thinking, complex reasoning, and written communications. 45+% of a sample of college students did not demonstrate any statistically significant improvement on the Collegiate Learning Assessment after two years of college. 36% of students did not show any significant improvement after four years. More disturbingly:

[We] find that learning in higher education is characterized by persistent and/or growing inequality. There are significant differences in critical thinking, complex reasoning, and writing skills when comparing groups of students from different family backgrounds and racial/ethnic groups. More important, not only do students enter college with unequal demonstrated abilities, but those inequalities tend to persist—or, in the case of African-American students relative to white students, increase—while they are enrolled in higher education.

An open letter to college admissions committees [Andrew F. Knight/Fairfax Times].

Consequently, the drive for high grades is blinding students and parents alike to the real purpose of education: learning. In parent-teacher conferences, “How can my child bring up her grade?” has replaced “How can my child better learn the material?” The system’s response to angry grade-obsessed parents and disgruntled students has been to fudge the indicator instead of improving the system in other words, to inflate grades in spite of worsening performance. I was routinely pressured by parents, students and even administrators to inflate grades in the form of curving scores, providing extra credit and retest opportunities, and more heavily weighting homework and projects that are easy to copy from friends. It is instructive to note that two-thirds of our students are on the honor roll. (That’s right.) When a majority of students routinely receive As and B’s in all their classes, the distinctions intended by a traditional A-F grading scale become hazy and meaningless.

What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study [Tamim, Bernard, Borkhovski, Abrami, & Schmid/Review of Educational Research]. A meta-meta-analysis of research on technology usage in education. Found random effects mean effect size of .35, statistically significantly different from zero. I have to wonder if that is meaningful in any way given the incredible variety of ways technology can be applied to learning. Have not read the full paper, only the abstract.

Health correlator: Calling self-experimentation N=1 is incorrect and misleading [Ned Kock/Health Correlator]. Self-experimentation is longitudinal, so n > 1. But results may not generalize to other people. Good for learning what works for you.