Government aims to build a ‘Data Eye in the Sky’

October 11, 2011 | Source: New York Times

Social scientists are trying to mine the vast resources of the Internet — Web searches and Twitter messages, Facebook, and blog posts, the digital location trails generated by billions of cellphones — to do what Isaac Asimov invented in his “Foundation” series: psychohistory, a new science that combined mathematics and psychology to predict the future.

The most optimistic researchers believe that these storehouses of “big data” will for the first time reveal sociological laws of human behavior,  enabling them to predict political crises, revolutions and other forms of social and economic instability.

This summer, the Intelligence Advanced Research Projects Activity (IARPA) began seeking ideas from academic social scientists and corporations for ways to automatically scan the Internet in 21 Latin American countries for “big data,” according to a research proposal being circulated by the agency.

The automated data collection system is to focus on patterns of communication, consumption, and movement of populations. It will use publicly accessible data, including Web search queries, blog entries, Internet traffic flow, financial market indicators, traffic webcams, and changes in Wikipedia entries.