Super Crunchers

People say that I have a good analytical mind; while I was always afraid of numbers, these people claim that I only feign fear. As a researcher, my creed has been of a qualitative persuasion. Friends take a dig at me as I argue that reality is socially constructed; I typically return the favor by saying that such socially constructed reality is statistically validated by the likes of my friends. I had, in the process of working on my morbid fear of numbers, undergone an internship with the analytics center of excellence in GE (GE-ACOE), Bangalore which later became a part of GECIS and is now a part of GENPACT. While I appreciated the way analytics were used to harness the power of numbers, I never really understood the import. Later on, the analytics industry grew and many people, including one of my good friends did their bit in evangelizing the analytics industry. Despite this, I failed to fully grasp the increasing importance of numbers in our life, till I read Super Crunchers.

Super Crunchers: Why thinking-by-numbers is the new way to be smart is an interesting book because its author Ian Ayres is primarily a law professor, and as such, he makes the book engaging by dealing in the applications of number crunching while keeping the math light at the same time. The book has a wide variety of interesting & informative illustrations with jargon reduced to a minimum. The technique dealt with primarily is that of randomization – the most interesting was the (unintended) experiment of reserving one-third seats of Panchayat heads in India to women, which led to establishing the difference in priorities of men and women heading villages. Indeed, the title of the book itself is a product of randomization, as Ayres tested at least 2 titles and 2 sub-titles before zeroing down on the present combination – surely one of the rare cases where an academician practices what he preaches? What is interesting is how he establishes the increasing feasibility of super crunching – thanks to advances in ICTs, the cost of setting up, data collection & analysis of large-scale randomization experiments has become simpler, faster and cheaper, leading to what he calls, a tad dramatically, “the end of intuition.” However, he does not elaborate on the negative effects of randomization – for example, large sample testing can have destructive effects, akin to testing the matches in a match box which become useless once tested. Also, intuition is still needed to decide on what to randomize. While Ayres argues that randomization cannot be used on once-in-a-while events like space shuttle launches, I was wondering if simulation, in combination with randomization, can help solve such problems. He also looks at regression and its applications in some depth in earlier chapters before moving on to introduce Bayes’ theorem and the 2SD rule (I never realized that the error margin indicated in polls is equal to two standard deviations) in the last chapter in an insightful yet enchanting manner. He delves into esoteric stuff such as neural networks and the counterintuitive strategy of limiting their number for achieving better results – taking me back to the time I learnt on a different conception of bounded rationality. Ayres looks to pause at the flipside too – the problems of errors in super crunching and the problems of reporters not understanding statistics; on the latter, he gives the example of Lawrence Summers’ statements being perceived as gender-insensitive and politically incorrect despite being statistically correct (Summers had to step down as Harvard president after the furor on the media coverage).

What I liked about the book was the flow in terms of how each seemingly disparate example seamlessly led to another, helping build arguments. Given the focus of the book, Ian did not dwell much on peripheral themes that I could glean (the much bigger role of government vis-à-vis private interests in randomization experiments; the increasing importance of number crunching in medicine, a theme touched upon in Malcolm Gladwell’s delightful Blink as well). He also does not explore some of the issues that stare you in the face as you read the book. He does not talk about the possibility of gaming behavior possible due to the results obtained through super crunching – when everyone starts engaging in such behavior, super crunching becomes a hygiene factor (a la “Does IT matter?” by Nicholas Carr); to me, the advantages of super crunching clearly lie in understanding (natural) systems that cannot be gamed. Despite these minor oversights on the part of the author, I really loved the book because it not only made me think but also made me feel as if I was in love with numbers!

Aside 1: I’d love to test which of these phrases is more effective using randomization: "100% improvement" or "increased by 2 times ."
Aside 2: I’d also like to write a sci-fi story on Super Crunching producing conformists.

© Author. All rights reserved.