Datasets In Mobile Apps Poses Challenge

on January 17, 2012 at 12:00 PM

Making federal government data more user-friendly and accessible to the public gives citizens an opportunity to ensure government accountability and encourages engagement and influence in the decision-making process.

According to the Mobile Gov Blog, has been a tremendous asset in accomplishing this goal, allowing anyone to download government datasets, large and small, However, while experts on the data can extract a wealth of information and publish their findings for everyone else to consume, the same large datasets can pose a particular challenge for developers of mobile apps.

Cost to Users of Including Whole Datasets in Apps

Most cell phone companies impose some sort of limit on their customers’ data usage. Using one provider as an example, the basic data plan has a limit of 200 megabytes (MB) per month. Once this limit is exceeded, the user can face additional charges. Another smartphone platform limits over-the-air app downloads to 20 MB to protect the users’ data caps.

The Department of Labor has many datasets published on that, if included in a mobile app, would consume at least a half of a typical smartphone user’s monthly data limit. For example, the Workforce Investment Act Net Impact Evaluation Dataset measures in at a hefty 321 MB!

One solution was to create an application programming interface (API) that would allow developers of web or mobile apps to download only the data that their app needs when it is needed. Rather than include the entire dataset, the app sends a request to DOL’s API asking for a much smaller subset of that data. The response is typically much smaller than an average webpage. While not a replacement for the datasets published to, the API provides instant, light-weight, and easy to access data for developers of web and mobile apps.

Initially launched with just three datasets, today APIs on provide access to 19 datasets containing 80 individual tables. One of the visions of our API effort is to ensure all of the department’s publicly available data is also available through the API, so expect the numbers to continue to grow.

DOL was not the first federal department or agency to make an API available to developers, but we were the first to provide software development kits (SDKs) and sample code to help developers make better use of our API. Our SDKs contain code that they can include in apps that take care of the connection to the API as well as making requests and retrieving data. This particular innovation lowers the barrier to entry, allowing someone with a great idea and basic programming skills to start developing apps with DOL data.

Our most recent innovation is our “API Sampler,” which allows developers to get a sample of what is returned by the API for testing and validation purposes. Links to the sampler are on the main page of as well as on each dataset’s page.

More innovations, enhancements, and datasets are in the works, so if you’re an app developer, keep your eye on DOL’s developer community website.