The future of the federal statistical system in an era of open government data was the subject of the recent Association of Public Data Users Conference (APDU). It gave me the unique opportunity to pose three questions about the ironic state of federal statistics to an august panel of experts.

The panel included: Connie Citro, Director of the Committee on National Statistics, National Academy of Science; Robert Groves, Director, US Census Bureau (until recently when he became the Provost at George Washington University, that allowed him to speak freely as you will see below); and Shirin Ahmed, Assistant Director for Economic Programs, US Census Bureau.

My three questions were:

Q: Why, when the current administration has spoken so much about data, hasn’t the federal statistical community leadership (e.g. OMB Chief Statistician Kathy Wallman) spoken up?

Answer: Because she is only an SES (Senior Executive Service member) and not a political appointee. (Connie Citro)

Q: Why didn’t a data agency like the Census Bureau get the job of hosting and managing Data.gov?

Answer: It fell victim to (former federal CIO) Vivek Kundra’s federal IT program and federal IT and statistics live in two different worlds. (Robert Groves)

Q. Why doesn’t the federal statistical community maintain FedStats.gov when my analyses shows that it still has better quality data and metadata than Data.gov does?

Answer: The current administration and political candidates are not talking about statistics and there needs to be a statafacts web site for this. (Robert Groves)

It is important to understand both the history of federal statistical data and the perception that statisticians have of open government data to understand the different worlds of the federal IT, Data.gov and the federal statistical communities FedStats.gov

I thought Connie Citro expressed it best with the title of her presentation: “The Federal Statistical System – A Crown Jewel, But Its History Makes It Harder to Meet Today’s Challenges.”

Shirin Ahmed articulated one of the key problems: There needs to be a legal basis for statistical agencies to share data and passage of the Data Synchronization Proposal by Congress will help.<

Then, Census Historian Margo Anderson, co-author of Encyclopedia of the US Census asked: Haven’t We Been Here Before? Historical Perspectives on the Federal Statistical System.

Anderson described the evolution from our basic founding principles to know who we are, to how are these data critical for job creation, poverty alleviation, and policy making. She described how the federal statistical system has periodically attempted to integrate administrative and operational records into the statistical system and how open data proposals before us now are different from the past.

Open data includes operational and administrative data used now by public agencies for statistical purposes. It’s changing the world of data users and producers alike.

Statisticians refer to their data as design data (data from survey’s designed to capture data that can answer questions with statistics with confidence levels) and open data as data that just turns up and becomes “big data” because it has no end, which a survey does, like the every 10 year census that lasts several months and then is done until ten years later.

So in answer to the question about bringing these different worlds of data together: Yes we have been there before, but it is definitely different from the past and in the words of Margo Anderson: “it is both brash and exciting.” Keep reading →

U.S. Chief Information Officer Todd Park talks about Health Datapalooza: A Model Of Innovation. The U.S. Census Bureau says Imagination at Work! Unleash Your Creativity With Our Census API.

Both deal with data. But which should it be: Innovation or creativity or both?

It’s question that deserves more than casual considering, and one I’m currently giving thought to for the upcoming Breaking Gov 38 Degrees Unleashing the Power of Government Data, Sept. 19, in Washington, DC.

It’s worth comparing the definitions. Wikipedia says: Keep reading →

FedStats.gov provides links to government data at http://www.fedstats.gov/qf/

Data.gov has been around for about three years now and is touted as the prime example of the Open Government Data Initiative based on its growth in number of data sets and communities using them. However, there have been two activities that have been around much longer, with more high-quality data sets, and a larger community, namely FedStats.gov and FedStats.net, which deserve continued attention in the government data community.

I was part of the FedStats Team that built FedStats.gov and led the FedStats.net Team. (You can read more about that team in a related story.) We received the Gore Hammer Award for that work to “Reinvent Government.” While Data.gov has helped focus attention on available government data, I see trying to reinvent that reinvention without the expertise that we had across the government at that time. The Data.gov Agency Points of Contact are not the same as the Federal Statistics Community. Keep reading →