Sunday, 18 November 2012

Of course it's the numbers, stupid...

Great interview in the Guardian with the data journalist's hero Nate Silver.
"Silver thought that by taking the available data and applying Bayesian theory to it, he might have "some small edge". There is a lot of data in American politics and Bayesian theory, a way of calculating conditional probabilities, has been around since an English clergyman, Thomas Bayes, first formulated it sometime at the start of the 18th century. It was not, as his critics have been quick to point out, exactly rocket science.
But then Silver is the first to agree. He's not even the only one doing it. Others had equally good results. At its heart, it's absolutely bog-standard statistics. He aggregates polling data. The twist or the "secret" to his model is what weight he gives to that data. How it's performed historically, what biases it might have, what other information can be brought to bear upon it. Yet, in the rarefied world of US politics, it's proved spectacularly more accurate than what was around before."
As this next little interview series on Germany's ZDF attests, the crossovers between coders and journalists, between data and investigation, are many and varied. The clip is in German, but there are a few non-Germans on the video including the Guardian's very own datablogger Simon Rogersdrone-journalist enthusiast Matt Waite and Propublica's Scott Klein. Oh, and me. Look away now...

No comments: