In this module, we read about cost-efficiency evaluations, which begins with an established goal or objective, and considers which among several approaches i

8

Cost-Efficiency Evaluations

W HEN ALL IS SAID AND DONE, WE NOT ONLY WANT CRIMINAL JUSTICE

policies to produce intended outcomes, like less crime; we also want the most “bang for the buck.” That is, we want policies that provide the most benefit for the least cost.

Most of us implicitly apply this logic in our day-to-day decision making. For example, we typically want a mode of transportation, such as a car, that reliably gets us to our destination, but we also want to pay as little as possible for it. Although the decision-making calculus may be complicated, the formula that we apply is not: among the available cars from which we can choose, we want the one that maximizes the benefit (e.g., reliable transportation) and minimizes costs (e.g., purchase price, repairs, fuel) over some set period of time.1

The formula applies as well when we have competing goals, but the cal- culations can be a bit more challenging. Perhaps we have a car that will no longer run. Then we learn that we have a dental problem that, if untreated, will lead to substantial pain. We need the car to get to work, but we also need surgery to live without pain. If we have the funds and no other competing demands, we could pay for both. Problem solved. However, many people face a real dilemma: buy a car or pay for surgery?2 Without a common met- ric for making a comparison between the qualitatively different outcomes (employment versus pain), we face a somewhat greater challenge in deter- mining which decision will yield the most benefit for the least cost. Even so, the focus on comparing benefits with costs remains the same.

Cost-efficiency evaluation involves this same type of logic and it comes in two varieties, one for each of the two situations just mentioned. Cost- effectiveness analysis is used to determine which of several approaches is best for achieving a given outcome, and cost-benefit analysis is used to determine which of several approaches that target qualitatively different outcomes

207

208 AMERICAN CRIMINAL JUSTICE POLICY

creates the most benefit relative to costs. To illustrate, law enforcement agencies must make decisions about which strategies most efficiently pre- vent crime. Cost-effectiveness analyses can be helpful in comparing such strategies. However, these agencies also must make decisions about other strategies that target other outcomes, such as efforts to reduce fear of crime and to provide emergency services that may save lives.3 Here, the outcomes (crime, fear of crime, lives saved) differ, and so we need a common metric for comparing the strategies that target them so that, in the end, we can compare which one produces the most benefit for the least cost.

Efficiency is of paramount importance to society and most organizations, especially when the risks are great (e.g., increased crime, fear, and injustice) and resources are scarce. That situation aptly characterizes criminal justice. For these reasons, cost-efficiency evaluation stands at the top of the evalu- ation hierarchy and is central to creating a more accountable and effective criminal justice system.

This chapter describes efficiency evaluation and the differences between cost-effectiveness and cost-benefit analyses. It focuses particular attention on the logic of such analyses and the basic steps involved in conducting them. It also highlights the central limitations and benefits of efficiency evaluation for criminal justice policy. The chapter then provides two case studies, one on community policing and the other on private prisons, to illustrate the idea of such evaluations. Paralleling the approach taken in previous chapters, it concludes by exploring the research foundation for assessing the efficiency of prominent U.S. criminal justice policies.

What Is a Cost-Efficiency Evaluation?

Cost-Efficiency Evaluation: Step 5 in the Hierarchy

THE EFFICIENCY MANDATE. The fifth and final step in the evaluation hier- archy involves cost-efficiency evaluation. Efficiency stands as perhaps the central guiding mandate of government – the aim is to provide services and achieve goals at the least cost and, more generally, to provide the maximum benefits to society. Achieving that mandate can be difficult when so many competing goals exist (e.g., education, public safety, roads) and when, in each instance, different approaches exist for trying to achieve them.

Regardless, the mandate exists and is embodied in virtually any taxpayer- funded effort. For example, when states allocate a certain amount of funds to corrections and not to education, they implicitly assume that the benefits of doing so exceed those associated with allocating the funds to education.

COST-EFFICIENCY EVALUATIONS 209

They may not conduct a study to test that assumption, but they nonetheless follow the logic. Similarly, when a department of corrections decides to build and operate a supermax prison, they assume that doing so produces benefits, relative to the costs, that exceed what could be obtained through other investments.

Despite policy makers’ calls for greater accountability, evaluations of the efficiency of public policies occur infrequently. Certainly, and as will be discussed later in this chapter, they occur rarely in criminal justice. The end result? Policy makers and criminal justice administrators perforce must make assumptions – ones that could be incorrect – about the relative benefits and costs of various policies, programs, and practices.4 This situation in turn creates substantial doubts about whether the government operates as efficiently as it could.

In the meantime, crime clearly creates substantial costs for society. As noted earlier, criminal justice expenditures in 2006 were $215 billion, a fig- ure that does not factor in the costs of victimization or other adverse effects of crime.5 In 1993, the last year for which a comprehensive assessment was conducted, the costs were estimated to be $450 billion.6 Many schol- ars dispute the validity of this estimate, noting, among other things, that they build on cost estimates associated with extreme rather than run-of-the- mill crimes.7 Even so, the expenditures alone and the dramatic increase in them over the past two decades highlight the fact that society has invested heavily in criminal justice and presumably expects some return on that investment.

COST-EFFECTIVENESS ANALYSIS VERSUS COST-BENEFIT ANALYSIS. Effi- ciency evaluations can consist of either cost-effectiveness analysis or cost- benefit analysis. Cost-effectiveness analyses identify the cost per outcome (e.g., for every dollar spent, one crime was averted). They are useful when compar- ing policies that target the same outcome. In the beginning of this chapter, the comparison of vehicles illustrated the logic of a cost-effectiveness analy- sis – the consumer aims to identify which vehicle will, over some period of time, provide the most reliable transportation for the least cost.

Cost-benefit analyses – sometimes referred to as benefit-cost analyses – identify, in monetary terms, policy costs and benefits (e.g., for every dollar spent, two dollars were saved). They are useful when comparing policies that target different outcomes (e.g., one aiming to prevent teen pregnancies and another aiming to reduce crime). The vehicle-versus-dental-care example at the beginning of the chapter illustrated the logic. In the example, the person attempts to identify whether achieving one goal (buying a car) will produce

210 AMERICAN CRIMINAL JUSTICE POLICY

more benefits than achieving another (obtaining dental care), relative to the costs of producing these benefits.

To illustrate further the distinction between the two types of analyses, consider the following two scenarios. First, you are the mayor a small town of 30,000 people, one that includes many of your best friends and family members. You have only $1 million to spend on efforts other than the bare bones minimum of paying for such services as roads, schools, and a medi- cal clinic. In the course of the past year, the number of aggravated assaults increased by 10 percent, from 100 to 110. You want to address the problem and task a committee with presenting several options. They convene and rec- ommend that you consider one of the following policies: longer prison terms for individuals who commit assaults (assume here that incarceration costs are paid for by the town), an employment training and placement program for at-risk populations, or a gang-reduction initiative. In each instance, the policy goal is the same – reduce assaults. Which policy do you select? Cost- effectiveness analyses would develop cost estimates for each policy and use information from impact evaluations of the policies. They then would create policy-specific estimated costs for, say, avoiding some number of assaults. The policy that costs the least to produce this result would be the most cost-effective.

Now, imagine a second scenario. Here, again, you are the mayor of a small town of 30,000 people and you have $1 million to spend on social problems. When you look over the past year, you see that assaults have increased 10 percent. However, you also see that teen pregnancies and the number of homeless people have both increased by 10 percent as well. To simplify matters, assume that in each case, a 10 percent increase amounts to ten additional cases (e.g., ten additional assaults, ten additional teen pregnancies, ten additional homeless persons). For the sake of argument, assume that impact evaluations have been conducted and show that, for $1 million, (a) a policy exists that will result in five fewer assaults in the coming year, (b) another policy will result in five fewer teen pregnancies, and, (c) a third policy will result in five fewer homeless persons. The investment cost in each instance is the same, but the outcomes differ. And therein lies the rub: an assault differs from a teen pregnancy and both differ from homelessness. What we need is a way to establish a basis of comparison. Monetizing – putting a dollar value on – a given unit of outcome enables us to establish comparability among the three outcomes. In turn, we then can create policy-specific estimates of costs versus benefits. The policy with the greatest return, as expressed in dollars, would be the most cost beneficial, all else being equal.

COST-EFFICIENCY EVALUATIONS 211

In some cases, cost-effectiveness analyses may suffice and have the virtue of sparing evaluators the difficult challenge of arriving at defensible mon- etized estimates of impact. The valuation of a life, for example, is a con- tentious issue.8 The problem lies in the fact that many criminal justice poli- cies and programs have multiple goals. Unless we factor all of them into an efficiency analysis, we end up with an unbalanced comparison. Enter cost-benefit analyses. They enable us to use a common metric – money – to compare a diverse set of outcomes. It might seem that cost-effectiveness studies would be suitable for a large swath of criminal justice initiatives aimed exclusively at reducing recidivism. However, different crimes have different costs. A rape, for example, typically would be presumed to produce more harm than theft of a bicycle.9 For this reason, cost-benefit analysis is use- ful, and indeed, necessary, for recidivism studies if we are to develop valid comparisons among policies.

How to Conduct a Cost-Benefit Analysis

Here, we will examine cost-benefit analysis and a number of conceptual elements central to undertaking one. The focus on cost-benefit analysis stems from the fact that the logic and steps involved parallel those for a cost-effectiveness analysis; the main difference is that cost-benefit analysis requires that we monetize all outcomes.

The focus on conceptual considerations stems from several considera- tions. First, it accords with a theme running throughout this book – namely, clear conceptualization is critical to undertaking and understanding policy evaluation, and, in many respects, it matters far more than the use of sophis- ticated research designs. That idea holds true with cost-benefit analysis as much as it does for needs, theory, implementation, and outcome and impact evaluations. Indeed, cost-benefit analysis, as Mark Cohen has emphasized, “is often more of an art than a science.”10 Second, an emphasis on con- ceptual considerations provides a more direct link to the discussions in the earlier chapters. For example, the veracity of a cost-benefit analysis rests heavily on a credible estimate of policy impact. Indeed, it can reasonably be said that a cost-benefit analysis is only as credible as that of the esti- mated impacts on which it rests.11 However, such estimates frequently may not exist, as Chapter 7 highlighted. By implication, any cost-benefit anal- ysis premised on an assumed impact should be viewed with considerable caution. Third, many excellent detailed accounts of the concrete nuts and bolts of cost-benefit analysis already exist and so do not need to be covered here.12

212 AMERICAN CRIMINAL JUSTICE POLICY

Many distinct tasks go into a cost-benefit analysis. Broadly, however, six steps need to be taken: (1) state the policy question, (2) identify the perspec- tive of analysis, (3) identify all relevant costs and benefits, (4) assign values to costs and benefits, (5) compare the costs and benefits of one or more poli- cies, and (6) assess the sensitivity of the results to critical assumptions and detail all relevant assumptions and limitations. Each step is described next.

1. STATE THE POLICY QUESTION. The relevant policy question and per- spective of analysis may seem obvious, but frequently they are not. Identi- fying them can require a great deal of conceptual footwork. For example, we might be interested, as a general matter, in whether the benefits of a particular policy, say, a supermax, exceed the costs. But such a question lacks specificity. In reality, we likely want to know if building a new super- max facility would be cost beneficial relative to some other alternative. Or we likely want to know if an existing supermax constitutes a cost-beneficial investment as compared to the costs and benefits of tearing it down and investing in a different type of facility or perhaps some other strategy or set of strategies for improving order and safety in a prison system. In short, the central challenge, at bottom, is similar to what we face when conducting or critiquing impact evaluations – identifying the relevant basis of comparison.

To make a useful determination about appropriate comparisons, we typ- ically need information about relative efficiency, that is, the efficiency of one approach as compared to another. To this end, it can be helpful to have a thorough needs evaluation from which to work. Such an evaluation can provide important context for describing the specific problems and the possible solutions to them. Perhaps, for example, a jurisdiction has experi- enced a jail-overcrowding problem. A needs evaluation might show that a large number of low-risk first-time offenders are being jailed prior to trial. If so, an efficiency analysis for a proposed jail might want to include not only the returns expected from constructing a new jail but also those that might accrue from a distinctly different policy option, namely, using jail space primarily for medium- and high-risk offenders.

2. IDENTIFY THE PERSPECTIVE OF ANALYSIS. Different perspectives exist – programs, government agencies, particular communities, cities and counties, states, and society, to name but a few – and each possesses a unique view that might well shape the specific questions posed in an efficiency evaluation. The societal perspective appears to be the one most frequently used. Government programs, for example, exist to help the public. Thus, their costs and benefits should be assessed with respect to society at large,

COST-EFFICIENCY EVALUATIONS 213

TABLE 8.1. The influence of the perspective of analysis on the classification of costs and benefits: An illustration using supermax prisons

Perspective of analysis

Department of Local Corrections Society community

a. Improved prison management Benefit – – b. Fewer in-prison assaults Benefit – – c. Improved postrelease success for

general population prisoners Benefit Benefit –

d. Weakened supermax inmate family relationships

– Cost –

e. Increased recidivism of supermax prisoners

Cost Cost –

f. Increased mental health problems among supermax prisoners

Cost Cost –

g. Increased domestic dispute incidents among prison staff and families of supermax prisoners

Cost Cost Cost

h. Additional property taxes Cost Cost Benefit

Source: Adapted from Lawrence and Mears (2004:10).

not some particular agency or group. However, cost-efficiency scholars dis- agree about the best or most appropriate perspective.13 Consider, for exam- ple, that a government agency charged with public health might reasonably ignore the crime-reduction benefits of drug treatment, while a criminal jus- tice agency might ignore the public-health benefits of reduced drug abuse.14

Why? When a department of corrections hires more staff, that investment comes from its budget not that of other state agencies. Accordingly, the department would likely include the additional staff as part of a cost-benefit analysis aimed at determining whether this investment or some other (e.g., purchase of equipment or technology) produced the most returns for some outcome of interest of direct relevance for the department (e.g., systemwide order and safety). In turn, any resulting cost-efficiency estimates of this investment might look appreciably different from those produced from dif- ferent perspectives of analysis (e.g., those of a department of public health).

To illustrate this point, consider, again, supermax prisons and the under- taking of cost-benefit analyses from three distinct perspectives – a depart- ment of corrections, society, and a local community.15 Table 8.1 presents

214 AMERICAN CRIMINAL JUSTICE POLICY

those perspectives and shows how some outcomes might be treated by each. Consider the first two outcomes, improved prison management (row a) and reduced in-prison assaults (row b). From a corrections perspective, both changes, potentially resulting from supermax incarceration, would constitute a benefit and so should be included in a cost-benefit analysis. However, from the perspective of society or a local community, neither outcome constitutes a benefit or cost. Neither outcome, for example, necessarily contributes to greater public safety.

When we turn to improved post-release success for general population inmates (row c), the picture changes slightly. Success might consist of lower recidivism rates, which a department of corrections would want to include in a cost-benefit analysis. From a societal perspective, lower recidivism clearly would stand as a benefit as well, and so from this perspective should be included in such an analysis. A benefit to the local community would be unlikely given that state prison inmates typically come from places other than the area where a given prison is located. However, should many of the released inmates return to that area, then a cost-benefit analysis from a local community perspective would want to include information about lower rates of recidivism.

Supermax housing might weaken inmates’ ties to their families and weaken or disrupt family relationships (row d). An effect on inmates’ fami- lies would not constitute a cost to the department of corrections. However, it might well constitute a cost to society, as depicted in the table.

If supermax incarceration were to increase rather than decrease recidivism (row e), then it would be a cost rather than a benefit, one that would be relevant from the perspectives of both the department of corrections and society. It also is possible that supermaxes increase mental health problems (row f), or so critics contend. Any increase would also count as a cost from these same perspectives.

The possibility exists that working in supermax facilities increases stress and domestic disputes among staff who work in them and the inmates placed there (row g). If such effects indeed exist, they likely would count as costs from each of the three perspectives of analysis.

Finally, we turn to an instance in which an outcome may be a cost or benefit. Increased property taxes (row h) may result when a department of corrections, and, by extension, society, invests in a supermax prison. These thus would be counted as costs from department of corrections and societal perspectives. However, for a local community, such taxes might well be treated as a benefit because of the increased revenues.

This example illustrates that no single or correct perspective of analysis exists. It also illustrates how the perspective of analysis can determine the

COST-EFFICIENCY EVALUATIONS 215

classification of outcomes as costs or benefits. In turn, it demonstrates the critical importance, when presenting or interpreting cost-efficiency results, of clarifying the perspective of analysis used and how the results might differ if other perspectives were used.

3. IDENTIFY COSTS AND BENEFITS. The third step in conducting a cost- benefit analysis entails identifying all relevant costs and benefits. As the discussion of step 2 indicates, the perspective of analysis will determine what is counted as a cost or a benefit and, more generally, what costs and benefits should be considered. A systematic and detailed theory evaluation can aid in this effort by highlighting a policy’s critical features, processes, and outcomes. Once we have selected the perspective and consulted a theory evaluation, the next step involves cataloging any cost or benefit related to the policy of interest.

In a cost-benefit analysis, a cost consists of anything that entails a loss and a benefit costs of anything that entails a gain, as expressed monetarily. To illustrate, expenditures – such as capital and operating costs associated with building and running a supermax prison – would be counted as costs, while improvements in outcomes (e.g., recidivism) would be counted as benefits. Whether a given item is classified as a cost or benefit ultimately depends, however, on whether it produces a (monetized) loss or a gain. For example, if a supermax prison increases recidivism, it creates a loss; accordingly, the increased recidivism would be classified as a cost. If, however, it decreases recidivism, it creates a gain or improvement; accordingly, the decreased recidivism would be classified as a benefit.

From an accounting perspective, several distinct types of costs and ben- efits should be cataloged. Numerous sources discuss these in detail, and Cohen and others have done so within a criminal justice framework.16 Con- sequently, here we will review only some of them to illustrate the conceptual and analytic work involved in undertaking and interpreting a cost-benefit analysis.

Costs and benefits can be classified in many ways, but one of the most important distinctions is between costs and benefits that are direct versus indirect. Direct costs and benefits “are those that are closely related to the primary objective of the [policy].”17 Direct costs of supermaxes, for exam- ple, include construction, personnel, technology, and other inputs, whereas direct benefits might include reduced violence and disorder throughout the prison system. Indirect costs and benefits “are by-products, multipliers, spillovers, or investment effects of the [policy],” what sometimes are referred to as externalities.18 Indirect costs can be intended or unintended. An in- tended indirect cost might include overhead associated with operating a

216 AMERICAN CRIMINAL JUSTICE POLICY

supermax and an unintended indirect cost might include increased sys- temwide violence or disorder that results from the use of supermax housing. An indirect benefit of a supermax might be reduced recidivism; it would be classified as indirect in states that expected no recidivism-reducing effect of supermax prisons.

Costs and benefits can also be classified as tangible and intangible. Tan- gible costs and benefits are those that we can easily identify and monetize, whereas intangible costs and benefits may be difficult to identify or mone- tize.19 Tangible costs might include, for example, victimization costs, such as medical expenses and lost wages. Tangible benefits might include reduced crime. Among released offenders, it might include improved physical or mental health, and, in turn, a reduced need for and use of social services or health care.20 Intangible costs might include increased pain and suffering or fear of crime and reduced quality of life, while intangible benefits might include decreased pain and suffering and increased quality of life.

Not least, costs and benefits can be classified as fixed and marginal. Fixed costs and benefits are constant, regardless of the size of or change in a policy. For example, if we purchase a video surveillance system, that cost remains the same whether we use it or not. By contrast, marginal costs vary depending on the size of or change in a policy or some feature of it. Should more drug- addicted offenders enter prison, more expenses may be incurred to provide treatment and thus would constitute a variable cost. Many criminal justice cost-benefit analyses use marginal capital and operating costs.21 Ultimately, the decision about what to include depends heavily on the specific focus of the cost-benefit analysis. For example, if a jurisdiction were deciding whether to expand drug treatment offered by an existing program, we would focus on marginal costs; most fixed costs would be irrelevant because the program already exists.22

Several observations about costs and benefits bear mention. First, dif- ferent types can be combined to help guide our efforts to identify all rele- vant costs and benefits. For example, there may be direct intangible ben- efits (e.g., a community policing program makes residents feel safer) and indirect intangible benefits (e.g., a community policing program helps pre- serve neighborhoods or facilitates efforts at revitalizing them).23 It may not always be easy to identify or classify them, but all such possibilities should be explored to ensure as comprehensive a listing of costs and benefits as possible.

Second, opportunity costs are an important consideration in cost-benefit analyses. “The opportunity cost of using a resource is the value of its next most valuable alternative use.”24 For example, an office might be needed to

COST-EFFICIENCY EVALUATIONS 217

support a particular crime-prevention initiative. If used in this manner, it no longer can be used in other ways. We cost the office, therefore, by measuring the value of whatever its next best use would be (e.g., its rental value).25

Unfortunately, it may not always be clear what the next-best use is or what value to include, which can create uncertainty in cost-benefit estimates.26

Also, what an opportunity cost is may depend on the perspective of analysis. How a prison system would have used funds for a supermax may vary, for example, from how state legislators would have used them.

Third, in developing a list of costs and benefits, we invariably will run into situations where we are not sure what to include.27 Here, an important rule of thumb can guide us: in general, we want to include only those costs and benefits related to the policy under consideration and that we would not otherwise occur.28 For example, if a supermax wing is added to an existing prison facility, we would include only the personnel costs associated with operating that wing, not the entire prison facility. The reason parallels the logic of impact evaluations – we want a clear assessment of a policy’s costs and benefits relative to what would have happened without that policy.

Fourth, the credibility of a cost-benefit analysis depends heavily on accu- rate cost and benefit estimates. For this reason, we typically want to proceed with an efficiency analysis only (a) if we have a credible estimate of policy impact or (b) if we want to explore how large an impact would have to be – and whether the magnitude of impact is reasonable to expect – to break even with our investment. If no impact evaluations have been conducted or those that exist rest upon a weak research design, then, by extension, we should have less trust in any efficiency estimates.

4. ASSIGN VALUES. Once all costs and benefits have been identified, we then want to assign monetary values to them. This step can be aided by recourse to data from agency records.29 However, it can be highly contro- versial because there may be little agreement about how to monetize some costs and benefits, especially intangible ones. Tangible costs and benefits “typically pass through a market system and have a price, such as articles of clothing or computer equipment.”30 In these cases, we can readily compile monetary values for each cost and benefit.

Intangible costs and benefits, however, typically have no market and thus no price. For example, no market exists for fear, pain, or suffering. Our option in this case is to proceed with a cost-benefit analysis that excludes such items. In so doing, however, we understate or overstate the true relationship between total costs and benefits. Our other option is to rely on imperfect estimates of intangible costs and benefits. The risk, again,

218 AMERICAN CRIMINAL JUSTICE POLICY

however, is that we understate or overstate the total costs relative to the benefits. No simple solution to this quandary exists, especially given that “the largest component of crime costs is quality-of-l