In a society where it seems nearly every move one makes can be tracked, it's not hard to see the easy comparisons to Big Brother in George Orwell's "1984". A quick dive into the world of big data only reinforces that sense, but it's big data that is powering the future of many government operations, particularly related to intelligence.
Across the U.S. government, particularly in defense and intelligence, agencies are implementing programs that take in huge volumes of data and turn it into actionable intelligence or predictions.
Much of that data is open source, coming from places such as social media. Programs track sentiments on Twitter, or complaints of not feeling well on Facebook. The result? Predictions of unrest or flu outbreaks.
That's the goal of programs like at least one under way at the Intelligence Advanced Research Projects Activity, where researchers use open-source indicators to automatically gather large volumes of open-source data and make predictions — much of the work done by machines.
"One area we looked at was disease events," such as predicting flu outbreaks, said Dr. Jason Matheny, Intelligence Advanced Research Projects Activity director. "We looked at a variety of indicators of people falling ill. One way is web searches of symptoms; another way is looking at cancelled dinner or flight reservations. Even though the data is anonymized, you can see if there are large numbers of those cancellations at restaurants or of flights. It allows you to take the pulse of the population and get an idea of an event happening in society."
During the Arab Spring, big data analysis was used to track Twitter use and predict looming uprisings. Now, a few years later, it's a similar story with a different, darker mission — though the Arab Spring and the Islamic State groupISIS, here, are inextricably linked.
At the Qatar Computing Research Institute in Doha, a team has developed an algorithm tracking Twitter use related to the Islamic State groupterror group ISIS. The database includes millions of Arabic-language tweets from more than 250,000 Twitter users, before and after the group's appearance of ISIS, according to a recent report from Britain's New Statesman.
"The resulting algorithm, says the team, can work out with 87 percent accuracy who is likely to support or oppose Isis in the future — before they've expressed any direct opinion online," the report noted.
According to the New Statesman article, the Defense Department is in talks with Arizona State University professor Paulo Shakarian, who also is leading an effort in predictive analytics based on ISIS activity.
RELATED: Open hybrid clouds create new savings and capabilities for military planners.
