Tax evasion or illegal drug smuggling are often not observable events for federal law enforcement officials. But to effectively manage federal law enforcement activities, officials and policy-makers in charge must have an idea of what is happening.

The challenge of how to measure the unobserved events is one faced by many federal leaders. But there are actually five methods that can assist government performance analysts in estimating basic information on unobserved events.
___________________________________________
This article originally appeared as part of a new report from the IBM Center for the Business of Government, “Five Methods for Measuring Unobserved Events: A Case Study of Federal Law Enforcement,” by John Whitley.

___________________________________________

The Need for a Statistical Framework

Law enforcement can face tough measurement challenges, but the fields of statistics and econometrics have developed a framework for dealing with them and it is useful to begin this part with a brief overview of that framework. All violations of a federal law can be thought of as elements of a prospective data population.

The scope of the population can be defined in various ways — e.g., immigrants illegally entering the United States in a calendar year, or the illegal drugs smuggled across the southwest land border between the United States and Mexico.

To effectively manage their operations, federal law enforcement officials need insight into these unobserved violations; i.e., they need to know the properties or parameters of this population of data, such as its size and distribution

Law enforcement officials are generally able to observe subsets, or samples, of this population. The most obvious is the subset of violators apprehended or arrested. Detailed documentation of apprehensions or arrests is generally retained in administrative records. In addition, there may be other available sources of data, often partial and incomplete, that shed light on various aspects of the population, e.g., survey data on drug usage or the footprints in the desert of illegal border-crossers.

Actions can also be taken to increase the available data, such as increasing the size of the observable subset, drawing additional samples from the population, or generating a sample of new data that mimics the characteristics of the population of interest. The methods described here use such samples to make estimates of the total population.

When using a sample to estimate parameters of the underlying (unobserved) population, an important statistical property is whether the estimate is biased. Bias occurs when the estimate systematically diverges from the true value of the population parameter being estimated.

An unbiased and therefore preferred estimate does not systematically diverge from the true value. One primary cause of bias is a poor sample that is not randomly selected. A sample is random when every element of the population has an equal probability of being included. Examples of non-random samples may include: