CIA

Intelligence Agencies Move Towards Single Super-Cloud

By

The intelligence community is developing a single cloud computing network to allow all its analysts to access and rapidly sift through massive volumes of data. When fully complete, this effort will create a pan-agency cloud, with organizations sharing many of the same computing resources and information. More importantly, the hope is the system will break down existing boundaries between agencies and change their insular cultures.

As in the rest of the federal government, lower costs and higher efficiency are the primary reasons for the intelligence world’s shift to cloud computing, said Charles Allen, formerly Under Secretary of Homeland Security for intelligence and analysis, currently a principal with the Chertoff Group, in an interview with Breaking Defense, an affiliate of Breaking Gov. Keep reading →

 /  Published December 17, 2012 at 12:01 PM

How In-Q-Tel Helps CIA Scout For Innovative Technology – A Model For Other Agencies?

By

US Government agencies often face a Catch-22 trying to adopt innovative technologies: Procurement rules designed to promote fairness can effectively preclude federal buyers from seeing – or influencing – developments that could eventually help agencies work more effectively.

The Defense Department and intelligence agencies, of course, have been fueling innovative technologies on their own for decades. But as commercial markets have exploded with new ideas, and learned to bring those ideas to market with greater speed, government agencies increasingly find themselves racing to keep up with innovations in the commercial sector. Keep reading →

 /  Published November 29, 2012 at 8:00 AM

Google Chairman Points To Dangerous Gap Between Culture, Technology

By


It didn’t take long before discussion about technology and national security Wednesday turned to this week’s Taliban violence on a Pakistani teenager.

“When reached by cell phone the Taliban claimed credit,” said Google Chairman Eric Schmidt referring to critically-injured Malala Yousufzai. “We know their phone number? Hmm. Why don’t we know where they are?” Keep reading →

 /  Published October 12, 2012 at 11:30 AM

Big Data Reaches The Hill: A Guide To Making It More Actionable

By

Big data, which has been the hot topic for conferences this year, has also received a good deal of attention on Capitol Hill in recent weeks, most notably with two recent events:


As one who represents a population of data scientists, a group for which the TechAmerica says there is growing demand, I have seen quite a few–and written a number of–articles about recent big data conferences:

For those who contributed to the ACT-IAC discussion with Congressional staff members on Big Data at the Hill – Defining and Understanding Policy Implications, I offer some specific ideas to three suggestions in their report:

What Congress should do to help big data Keep reading →

 /  Published October 10, 2012 at 8:00 AM

Defense Department Bets Big On Big Data

By

First Todd Park, former Department of Health and Human Services chief technology officer, bet on health data in a big way; got his upcoming Health Data Palloza, and then became our new Federal CTO.

Then Gus Hunt, CIA CTO, bet on big data for the Intelligence Community and got its budget increased by Congress, reflecting a governmental shift in IT priorities, from a Defense Department style network-centric focus toward the IC’s big data-centric focus.

Now the Defense Department is in the big data game with their big bet to the tune of $250 million announced Thursday at the White House Office of Science and Technology Policy’s Big Data Research and Development Initiative.

The assistant secretary of Defense, in a letter released yesterday, said “We intend to change the game and plan to be the fist to leverage big data across the full scope of military operations in new and unconventional ways.”

There are five other agencies who were present at the AAAS Auditorium event which are contributing much smaller (or non-disclosed amounts) as follows:

  • National Science Foundation: $10 million, plus several smaller grants
  • DARPA: $25 million annually for four years
  • National Institutes of Health: No money, but the world’s largest set of data on human genetic variation freely available
  • Department of Energy: $25 million
  • USGS: New grants for unspecified amounts
But where does this new initiative leave us?

I think it leaves us with a disconnected federal big data program between the science and intelligence communities with the former considerably behind the latter.

The report, “Designing a Digital Future: Federally Funded Research and Development in Networking and Information Technology,” prepared by the President’s Council of Advisors on Science and Technology (PCAST), said: “Every federal agency needs to have a “big data” strategy.

I did not hear that today either from every agency or across all the agencies. The recent 2012 Big Data Government Forum provided a much more comprehensive view of best practices around Big Data technology, trends, and issues from senior government executives, data scientists, and vendors.

As Professor Jim Hendler, RPI Computer Scientist, commented during the meeting: “Computer scientists like us have to move to the social science side of things to really do big data.”

This new White House Initiative needs Todd Park’s entrepreneurial spirit, Gus Hunt’s experience, and DoD’s new money, spent in a coordinated way with the IC and civilian agencies to make big data across the federal government a reality.

 /  Published March 30, 2012 at 3:58 PM

Why The Intelligence Community Loves Big Data

By

The fact that the Department of Defense got its budget cut and the Intelligence Community got its budget increased in the White House’s 2013 budget request of Congress is indicative of more than the need to roll back a decade of military growth. It’s also indicative of a shift in IT focus–and a reflection that DoD’s network-centric focus is being overtaken by the IC’s big data-centric focus.

There are probably many reasons for such a shift. One is the world’s population. The U.S. Census Bureau estimates the world population passed 7 billion mark this past weekend. The rapidly growing number of people who will eventually have smartphones with multiple sensors (your iPhone has them now for GPS position, etc.) promises a future where there will be massive streams of real-time data that the IC will want to mine, looking for lone-wolf terrorists (who are relatively but easy to stop) who I have written about previously.

For companies like Google and Facebook, big data is big business, and for other companies big data is becoming their business as they mine their large swaths of data to improve their services and develop new business activities. The IC may not come out and say it, but it has to love the fact that Facebook will soon have 1/7th of the world’s population using it’s platform to share what’s going on. Or that Google is almost everyone’s favorite search engine because they can keep track of what people are posting and searching for much easier than many in government can.

The IC also has to love big data, and the rapid evolution of systems used to ingest and process it, because it helps push the technology wave, as Gus Hunt, CIA chief technology officer (pictured above), described it at the recent Government Big Data Forum.

Hunt said that in every aspect of their workflow at the CIA, from sensors to finished intelligence, massive, multiple, real-time sensor data streams cause bottlenecks on current networks that swamp current storage devices and overwhelm current query, analytics, and visualization tools, that are needed to produce finished intelligence.

So he wants his cake and to eat it too: He wants real-time analytics and visualizations that he says a few start-ups are trying to achieve. He also wants the Federal Cloud Computing Initiative to add two more services to Platform-, Software-, and Infrastructure-as-a-Service, namely, Data-as-a-Service and Security-as-a-Service.

Part of the solution is emerging from Google’s MapReduce, which is a parallel data processing framework that has been commercialized as Apache Hadoop (developed by Doug Cutting who named it after his son’s toy elephant) by Cloudera so one can store and compute big data at the same time.

Amr Awadallah, founder and CTO of Cloudera, calls Apache Hadoop a data operating system in contrast to Windows and Linux, which are essentially file operating systems (they store and manage all the files you create and are needed for your software applications). He points out that Apache Hadoop provides the three essential things: velocity, scalability, and economics, that are needed to handle big data.


So the IC, Gus Hunt, Amr Awadalla, and others at the Government Big Data Forum are leading the next technology wave and gave us a glimpse of both the technology infrastructure and the business organization with chief data officers and data scientists that will be needed to implement and succeed with big data.

More details about what was said can be found at CTOVision and at my wiki document, Data Science Visualizations Past Present and Future.

 /  Published March 13, 2012 at 9:30 AM

FCC Chairman Warns Hackers Gaining Means To Shut Down The Internet

By

Federal Communications Commission Chairman Julius Genachowski today called upon the nation’s Internet Service Providers (ISPs) to “take concrete steps” to improve Internet security for consumers and critical infrastructure, warning that a failure to do so could slow broadband adoption and threaten the nation’s economy.

With more than $8 trillion exchanged electronically every year, Genachowski warned that sophisticated hackers are gaining the expertise to “shut down the Internet…shut down our economy [and] compromise our growth engine.” Keep reading →

 /  Published February 22, 2012 at 4:42 PM

CIA’s World Factbook: Intelligence For Every Thinker, Traveler, Soldier, Spy

By

The Government Printing Office has just recently released its latest edition of the CIA’s World Factbook–which marks its 50th anniversary in 2012 for the classified version and more than 40 years of publishing the public version.

The 810-page public edition of the CIA’s World Factbook provides not only a timely and valuable source of global information, it also allows us a glimpse into the times and events that necessitated its production. Keep reading →

 /  Published February 2, 2012 at 8:09 AM

Lessons From Army’s Weapon Systems Handbook – And Gall’s Law

By

The 2012 Army Weapon Systems Handbook is available in a new, easier to access format.

I know, because Terry Edwards, Director of the Army’s System of System’s Engineering (SoSE), asked me to do it, but more to the point, I learned some lessons from the previous version (simpler is better) that I want to share with readers, especially those that want to build their own dashboards.

First some background: “The Army Acquisition Executive has launched a new highly collaborative SoSE campaign aimed at synchronizing development and delivery of technologies across the entire Army systems portfolio, service officials have said.

“Among the effort’s central tenets is a need to align programs more closely and establish an acquisition strategy that draws simultaneously from programs of record (PORs), commercial-off-the-shelf products, and emerging technologies from the Army’s Science and Technology Directorate–all as a way to maximize efficiency across the Army’s developmental spectrum.”

That led to a dashboard which is an important step to implementing Dynamic Case Management, such as Be Informed 4, and Business Events, such as TIBCO Solutions for the Army SOSE.

I first decided what this dashboard should be about: What should be Linked Open Data be like and what should be Structured Data be like. I concluded they should be similar to work done with the CIA World Factbook recently.

This is really important to the Quint (CIA, DIA, NGA, NRO, and NSA). Do not worry about what all those acronyms mean, just that they keep us safe, especially if they all work together “to connect the dots” and connecting the dots involves connecting unstructured and structured data by making unstructured data Linked Open Data as a first step.


Leaving out most of the details, I copied the Army Weapons Systems Manual table of contents to my wiki-scraper tool and gave it well-defined web addresses and drilled down within each item to give it additional well-defined web addresses (not to short and not too long). Boring, but absolutely essential work to succeed. That all goes into a spreadsheet which gets imported to a dashboard tool where the data sets can be sorted, searched, merged, etc. The detailed results are shown elsewhere.

One can go from the DoD System of Systems to the Army Weapons System of Systems to the individual systems. Now what you really want to do is use that to manage an enterprise of 153 weapons systems efficiently and effectively as Terry Edwards described above, in his work which is described further at “Army’s Resource Forest Is Good Metaphor For System Of Systems Approach.” Keep reading →

 /  Published December 29, 2011 at 9:03 AM

What Government Can Learn From Moneyball, Baseball Stats And Albert Pujols

By

Recently, Shelley Metzenbaum, associate director of performance and personnel management at the Office of Management and Budget, blogged about Saving Taxpayer Dollars With Moneyball.

She said: “Using all the relevant data we can find to do more with less must be the rule, not an exception, in government.” Keep reading →

 /  Published December 2, 2011 at 3:32 PM
Page 1 of 212