Uncharted: Big Data as a Lens on Human Culture

Uncharted: Big Data as a Lens on Human Culture, Erez Aiden and Jean-Baptiste Michel, Riverhead Books, 2013

Everyone has heard of Big Data; huge amounts of information, usually involving computers or the Internet. Is there a cultural or historical equivalent of Big Data?

Yes, and it comes from Google's intention to digitize all the world's books (or, at least, a significant portion of them). The authors created an algorithm that would search all those books for certain words. On a chart, it will show, for instance, how many times, per million words, the name "Abraham Lincoln" was used, or "World War II." It can also be used to compare the historical use of pairs of words, like Satan/Santa, evolution/DNA, men/women, war/peace, tea/coffee or old school/new school. It can be found at books.google.com/ngrams ("Possibly the greatest time-waster in the history of the Internet." - Mother Jones magazine). Google needed convincing that this was a good idea, that it would not open them up to millions of copyright infringement lawsuits.

Using this algorithm, it is possible to look at things like historical attempts at censorship. It can range from Nazi attempts to remove Jewish artists like Marc Chagall from the German cultural landscape, to the 1950's Hollywood Blacklist. A person can also look at how long a certain word or phrase stays in the cultural memory. For instance, "Korean war" has a huge jump in usage in the 1960's, then an equally huge drop in usage soon after, down to its present level of almost nothing.

The book also looks at the evolution of the English language. If we have pairs of words like drive/drove, what happened to thrive/throve? Also, what happened to words like burnt, learnt and dwelt? It all has to do with irregular verbs, which change over time.

This is a fascinating book, but it will take some effort on the part of the reader. It's very well done, and it gives the reader the chance to do their own historical research.