The Economist on the data deluge

Been waiting for this week’s Economist ever since I heard that it included a special report on managing information. Got it today.

Three things I liked in the first article Data, data everywhere

1. We need to train people to work with data.

Alex Szalay, an astrophysicist at Johns Hopkins University, notes that the proliferation of data is making them increasingly inaccessible. “How to make sense of all these data? People should be worried about how we train the next generation, not just of scientists, but people in government and industry,” he says.

When will K-12 get the clue that top math students (really, all math students) need to learn probability and statistics in addition to or possibly in place of calculus? To be fair, the latest high school math standards I’m familiar with do include some coverage of probability and statistics. But at the AP level, the calculus class is much more rigorous and more respected than statistics. Yet many students would be better off learning statistics.

2. Statistics is sexy.

a new kind of professional has emerged, the data scientist, who combines the skills of software programmer, statistician and storyteller/artist to extract the nuggets of gold hidden under mountains of data. Hal Varian, Google’s chief economist, predicts that the job of statistician will become the “sexiest” around. Data, he explains, are widely available; what is scarce is the ability to extract wisdom from them.

oh yeah!

3. Social science is ripe for revolution.

More data and better techniques mean we can understand human behavior at every level better than ever before. 

“Revolutions in science have often been preceded by revolutions in measurement,” says Sinan Aral, a business professor at New York University. Just as the microscope transformed biology by exposing germs, and the electron microscope changed physics, all these data are turning the social sciences upside down, he explains. Researchers are now able to understand human behaviour at the population level rather than the individual level.