In Nate Silver’s world where data drives predictions, underdogs sometimes have the advantage, crises arising from natural disasters can be averted and shocking electoral victories like President Donald Trump’s aren’t surprising at all.
Silver, the editor-in-chief for ESPN’s FiveThirtyEight website, gave a Texas A&M audience an opportunity to see – and predict – the world the way he does by challenging them to make decisions probabilistically using big data to supplement instinct and experience.
In his discussion presented by MSC Bethancourt and titled The Signal and the Noise: An Evening with Nate Silver, he outlined why the world hasn’t become a more predictable place despite large scale data accumulation and how big data can be a guide for more accurate decision making and better outcomes.
For Silver, big data can sometimes drown people who try to use it in information. The solution, he said, is striking a balance. Instead of using instinct to make a decision and using data to justify that decision, let data chart the course and incorporate common sense and experience to handle the rest.
“If you’re betting on a correlation and you don’t know the reason behind it, I’m usually going to take the other side of that bet,” Silver said. “Brute force data accumulation that isn’t corrective learning and doesn’t apply the scientific method can be a really dangerous approach sometimes.”
To underscore this point, he used a story from when he took a taxi to a speaking engagement in New York City and the driver took a winding path to get to a destination at the demand of his GPS. Instead of using the city’s intuitive numbered grid street system, the driver was directed to cut across Central Park twice and was 45 minutes late to his destination. The driver, Silver said, put too much faith in the technology and data, instead of using his intuition at the crucial points.
“Yeah, in the long run having more data and more information is better, but there is a whole host of techniques that we as a society need to learn,” Silver said. “I’m interested in how people react when they have an algorithm or a formula on the one hand and intuition or experience on the other hand.”
