The Great Data Evolution

Human nature likes a hero, maybe it even demands it. Good storytelling can bring a mundane or seemingly uninteresting topic to life. So when Michael Lewis was able to bring a story of statistics at a mediocre baseball team to world-wide recognition you know it’s a good story. Inadvertently or otherwise Lewis paints Bill James as the father of Baseball statistics and Beane as a willing and capable disciple. You hear endless conversations refer to the data revolution beginning with Bill James, much like Sky would have you convinced that football began with the Premiership.

download (1)A recent article I read and loved, by Joe Posnanski, titled; VANGUARD AFTER THE REVOLUTION ads fuel to two fires. First that what happened in Baseball was a revolution and second that Bill James started it. When you delve into the history of baseball statistics, as this book so brilliantly does you don’t find a revolution. Far from it in fact. You find a slow (perhaps painfully at times) crawl towards what we see today. Far from being the revolutionary, Bill James was a significant torch bearer for something that began long before him.

Dig into Baseball’s past and you will find articles dating back to 1910 titled; ‘Where the Baseball records fail to tell the truth’, ‘A brand new system for batting average’ & ‘A square deal for the Base on Balls’. With zero technology and unofficial scoring methods pioneers such as FC Lane and Henry Chandwick among others were questioning, probing, analysing!

And it wasn’t just keyboard warriors (pencil and paper warriors?) that were interested. As early as 1913 people were collecting data on a commercial basis. Two brothers Al Munro & Walter started collecting ‘data’ and selling it. That company (Elias Stats) still exists today and remains the official statistics provider to the MLB, NFL, NBA, MLS & WNBA. That’s some list of clients, but maybe a case for evolution not revolution?

Major League Baseball even had it’s first full-time analyst employed in 1947. Alan Roth was the Brooklyn Dodgers numbers man. “As vital to Brooklyn boss Branch Rickey as Robin was to Batman”.

The first full-time statistician ever hired by a major league club, Roth spent his days churning out figures that no one had ever seen before: runners advanced and bunting success rates; batting averages in every ball-strike count; and performance against lefty and righty hitters or pitchers. … Every one of the 40,000 pitches in Brooklyn’s season called for up to a dozen notations onto Roths yellow legal pad, all so that he could discover any little edge the Dodgers could exploit”

No doubt today’s technology is different, the algorithms unrecognisable and the yellow legal pads long discarded but the essence is exactly the same… Any little edge the Dodgers could exploit.

numbers-gameThere were even statistical books written about Baseball. In 1964 Earnshaw Cook wrote Percentage Baseball, the first accessible baseball statistics book. And there were more to follow, some were encyclopaedias of stats, other with more narration. And of course Bill James contributed more than admirably to the collection.

Far from being a story that starts with Bill James – baseball’s statistical history looks much more like evolution. Some great minds, mixed with improving technology, increased commercialization of sport, increasing player values and the higher cost of mistakes all form this giant melting pot that make it seem like a revolution took place. Sure it makes for a nice story, stories need heroes and Bill James, Billy Beane, Nate Silver and even Brad Pitt play good heroes but that isn’t really the truth. What we are witnessing is evolution not revolution, yes it’s been slow and now it seems faster but maybe in 100 years this time will look slow and archaic too.

Football (media) is perhaps still waiting for it’s hero. Brentford are likely contenders along with Fc Midtjylland or maybe Billy Beane at Az Alkmaar. One thing is for sure though, there are countless analysts, bloggers and visionaries who will laugh at the idea that this is only starting now.

P.S: The Numbers Game by Alan Schwarz would make it into my top 10 books of all time. It won’t be everyone’s cup of tea but I loved it. 

Add Comment