Is ‘Big Data’ the next big thing?

August 13, 2012

A data visualization created by IBM shows that Big Data such as Wikipedia edits by bot Pearle are more meaningful when enhanced with colors and position (credit: viegas/Wikimedia Commons)

This has been the crossover year for Big Data — as a concept, a term, and a marketing tool, as it enters the mainstream, The New York Times reports.

Big Data was a featured topic this year at the World Economic Forum in Davos, Switzerland, with a report titled “Big Data, Big Impact.” In March, the federal government announced $200 million in research programs for Big Data computing.

Big Data is a shorthand label that typically means applying the tools of artificial intelligence, like machine learning, to vast new troves of data beyond that captured in standard databases. The new data sources include Web-browsing data trails, social network communications, sensor data, and surveillance data.

In theory, Big Data could improve decision-making in fields from business to medicine, allowing decisions to be based increasingly on data and analysis rather than intuition and experience.

The Data Collective is a fund specializing in investments in companies doing everything from creating large, complex databases and faster processing of information from diverse sources, to applications that make use of novel and interesting patterns that might be exploited, another Times article reports. It already includes 35 equity partners who have a lot of experience in the field, and the fund has already made 46 investments.

And Smart Data Collective suggests five scenarios for how relationships between data scientists and IT could evolve.