Director of National Intelligence

Federal leaders for government-wide acquisition and information-sharing initiatives have joined forces with technology suppliers to hammer out a new set of recommendations to identify and use the government’s information sharing standards and requirements.

The goal of the recommendations is to enhance national security, increase efficiency and reduce costs by improving collaboration between government and industry in developing open interoperability standards and incorporating them into commercial products Keep reading →


The Obama administration is getting ready to change the way the government handles cybersecurity.

The White House has drafted an executive order, a draft of which is currently circulating among federal agencies for approval, mirroring cyber legislation that recently failed to get through a Senate vote. Among other things, the order shunts much of the enforcement and management of cybersecurity issues to federal agencies. We understand that, contrary to some earlier news reports, the classified portion of the order does not contain significant new authorities but details those already existing. Keep reading →

“If people do not emit or discuss their behavior, it’s hard to find out what they are going to do,” declared Director of National Intelligence Jim Clapper, speaking recently at the huge annual conference of intelligence professionals called Geoint.

The U.S., he made clear, did not have tactical warning of the Benghazi attacks. He also noted that anti-American protests were taking place in 54 countries when the attacks occurred, clearly implying the intelligence community was hearing a lot of noise that day.

While it’s true, it’s hard to predict what people will do, it’s also increasingly true that people worldwide do emit and discuss their behavior through online social media and that can be monitored and analyzed in nearly real-time.

Companies such as intelligence analysis contractor Recorded Future, which is backed by Google, the CIA, and others for this very reason, offer examples of that on a routine basis. Recorded Future, for instance, tracks protests occurring around the world by extracting references to those events and their locations from online media.


Evidence offered at the first ever Recorded Future User Network (RFUN) conference at the Newseum in Washington, DC, and by a recent Washington Post story, clearly suggest that protests are rising worldwide, according to this visualization produced by the Recorded Future.

One of their measurements records “intensity” of references to protests in online media and social media, with color coded indicators signifying heavy discussion of protests, little or no discussion at all. One can also see the list of terms associated with protests in a selected country during the selected week and protests planned for the future.

There are certainly caveats to be kept in mind with these data sets and visualizations. The protest metric is not perfect, and not all protests are comparable. Much has been made of the role of social media in facilitating protest movements in the Middle East and China, but it could also be amplifying their impact.

Dr. Melissa Flagg, a senior Department of Defense manager at the RFUN event noted how analysts are starting to use Recorded Future to guide future DoD science and technology investments by tweeting: “We have to tell the story of the long term to the people who only care about the next three minutes.”


Recorded Future CEO Chris Ahlberg and inventor of Spotfire, says that Recorded Future is the only comprehensive source of past, planned and predicted events on the web and the world’s first Temporal Analytics Engine. His goal is to eventually provide access to everything on the Web in nearly real-time. This of course would be an example of really big data analysis, using the cloud like the National Security Agency has talked about.

Recorded Future is also stressing the importance of data science and data scientists in all of this. Their event featured Drew Conway (@drewconway) speaking about the joys, challenges, and power of data science. Among other insights, Conway showed some results that suggest the peaks in country protests are related to food shortages.

As a data scientists having also worked with Recorded Future on things like Visualization Of The Osama Bin Laden Letters, I could not resist checking and building on their analysis of trends in global protests (see table below.)

It shows, for instance among the top 10 countries for protest intensity, registered in the Recorded Future data set for October 1, 2011, to October 11, 2012, the United States is the most frequently mentioned term – showing up in five of the top 10 instances.

The chart also includes comments worth noting.

Keep reading →

The convergence of several rapidly evolving technologies is creating new potential for innovation at federal agencies, a group of senior government officials said at a technology and innovation forum held in Washington, D.C., April 24.

The accelerating adoption of cloud computing strategies, the consumerization and commoditization of IT, the integration of mobile devices and applications in the workplace, the rise of social media, and the need to process exponentially greater volumes of data are each unleashing new and more cost effective ways to work, the officials said. Keep reading →


Women in government technology provide much more than a shift in statistics. The diversity provides a hotbed for innovative ideas, top female executives said during a keynote session at the annual FOSE convention in Washington, D.C., on Thursday.

“There’s a real business reason to think about how we bring more women into the workforce,” said Lisa Schlosser, deputy associate administrator in the Office of Management and Budget’s Office of E-Government and Technology. “The fact of the matter is that bringing diversity into the workplace inspires more ideas and innovation.” Keep reading →


The Federal government is now on track to close 1,080 data centers by 2015 among 3,133 in operation as part of a broader administration effort to reduce duplicative spending and to do more with less.

Federal CIO Steven VanRoekel, however, has signaled a new emphasis on doing “more” — by investing in technology creatively–rather than focusing primarily “on the less” that is typically associated with cuts, including data center closures. Keep reading →

It is clear to me that the CIA needs big data, like Zettabytes (10 to the 21st power bytes), and the ability to find and connect the “terrorist dots” in it. As of 2009, the entire Internet was estimated to contain close to 500 exabytes which is a half zettabyte.

Recently I have listened to three senior CIA officials — two former and one current — talk about this and the need for data science and data scientists to make sense of it.

Gen. Michael Hayden, former director of the CIA and National Security Agency, and Principle Deputy Director of National Intelligence, and Bob Flores, former chief technology officer at the CIA, spoke about this at the MarkLogic Government Summit; and Gus Hunt, current CTO at CIA, spoke about this at the Amazon Web Services Summit that I wrote about recently.

General Hayden framed the problem as follows: Cold War Era — easy to find the enemy, but hard to stop them (e.g. Soviet tanks in Eastern Germany); versus the Global War on Terrorism — hard to find the terrorist, but easy to stop once their found (e.g. the underwear bomber on the airplane). He said we live in an era where it is not a failure to share data, but with processing the shear volume and variety of data with velocity that is the result of sharing.

He shared his experience meeting with former Egyptian President Mubarak before the recent Arab awakening due to social media that resulted in his overthrow and then meeting with the President of Twitter, Jack Dorsey, whom he asked: How does it feel to overthrow a government–something the CIA, when Hayden was director, was never able to do?

Hayden also said we need tools to predict the future from social media and data scientists to use them.

I told him about my work with Recorded Future that was also the subject of an Breaking Gov story.

Bob Flores, former CIA CTO, said that Recorded Future was a new, fantastic technology and that the old model of collect, winnow, and disseminate fails spectacularly in the big data world we live in now. He used the recent movie “Moneyball” as an example of how the new field of baseball analytics called Sabermetrics has shown there is no more rigorous test (of a business plan) than empirical evidence.

He said that in this time of budget cuts and downsizing the creme will rise to the top (those people and organizations can solve real problems with data) and survive. And Flores agrees with Gen. Hayden that while all budgets are on a downslope (including for defense, intelligence, and cyber), that cyber is on the least down slope of all the rest because it is realized that limiting the analysis of big data would be equivalent to disarmament in the Cold War era.