Fluent Essays

Read the scenario:As a legislator in Washington State, you have just read, Evidence-Based Public Policy to Reduce Crime and Criminal Justice costs” implications in Washington State, and you must determine what the best course of action is for the State to pursue as far as building new prisons.Answer the following:What are you going to recommend and why?If you determine building additional prisons is not in the best interest of the State, how will you “sell” your recommendations to the public without appearing “soft” on crime? wsipp_evidence_based_public_policy_options_to_reduce_crime_and_criminal_justice_costs_implications_in_washington_state_full_report.pdf Unformatted Attachment Preview Victims and Offenders, 4:170–196, 2009 Copyright © Taylor & Francis Group, LLC ISSN: 1556-4886 print/1556-4991 online DOI: 10.1080/15564880802612615 Evidence-Based Public Policy Options to Reduce Crime and Criminal Justice Costs: Implications in Washington State 1556-4991 1556-4886 UVAO Victims and Offenders Offenders, Vol. 4, No. 1, November 2008: pp. 1–35 Public E. K. Drake, Policy S. Options Aos, and M. G. Miller Elizabeth K. Drake, Steve Aos, and Marna G. Miller Washington State Institute for Public Policy, Olympia, Washington, USA Abstract: In 2006, long-term forecasts indicated that Washington faced the need to construct several new prisons in the following two decades. Since new prisons are costly, the Washington legislature directed the Washington State Institute for Public Policy to project whether there are “evidence-based” options that can reduce the future need for prison beds, save money for state and local taxpayers, and contribute to lower crime rates. The institute conducted a systematic review of all research evidence that could be located to determine what works, if anything, to reduce crime. We found and analyzed 545 comparison-group evaluations of adult corrections, juvenile corrections, and prevention programs. We then estimated the benefits and costs of many of these evidence-based options and found that some evidence-based programs produce favorable returns on investment. This paper presents our findings and describes our meta-analytic and economic methods. Keywords: cost effectiveness, correctional intervention, evidence-based policy During the mid-1990s, the Washington legislature began to enact statutes to promote an “evidence-based” approach to several public policies. While the phrase “evidence-based” has not always been precisely defined in legislation, it has generally been constructed to describe a program or policy supported by outcome evaluations clearly demonstrating effectiveness. Additionally, to determine if taxpayers receive an adequate return on investment, the legislature began to require cost-benefit analyses of certain state-funded programs and practices. Address correspondence to Elizabeth K. Drake, Washington Institute for Public Policy, PO Box 40999, Olympia, WA 98504. E-mail: ekdrake@wsipp.wa.gov 170 Public Policy Options Washington’s initial experiments with evidence-based and cost-beneficial public policies began in the state’s juvenile justice system. The legislature funded several nationally known and well-researched programs designed to reduce the reoffending rates of juveniles. At the same time, the legislature eliminated the funding of a juvenile justice program when a careful evaluation revealed it was failing to reduce juvenile crime. Following this initial successful venture into evidence-based public policy, Washington began to introduce the approach to other areas including child welfare, mental health, substance abuse, K–12 education, and adult corrections. In 2005, long-term forecasts indicated that Washington would need two new prisons by 2020 and possibly another by 2030. That year’s legislature directed the institute to determine if evidence-based options existed that could reduce the need for prison construction, save money for state and local taxpayers, and contribute to lower crime rates (Capital Budget, 2005). We conducted a systematic review of all the research evidence we could locate in adult corrections, juvenile corrections, and prevention programs and found that some evidencebased programs reduce crime while others do not; we also conducted an economic analysis of many of the programs (Aos, Miller, & Drake, 2006). Based on the findings, the 2007 legislature made significant investments by allotting $48 million in the biennial budget for the expanded use of evidencebased programs. Investments were made in many adult and juvenile justice programs, as well as in prevention programs—including drug treatment, education, vocational training, correctional industries, functional family therapy, multisystemic therapy, aggression replacement training, and early childhood education. The state’s prison forecast was subsequently adjusted downward to reflect the resource decisions made by the 2007 legislature. In this paper, we present the findings from our 2006 study, including some revisions since its publication. This research is part of an ongoing effort to improve Washington’s criminal justice system; the narrative presented here is a snapshot of the current analytical process. Due to space limitations, we focus on our statistical review of the evaluation literature and on our per-program economic analysis. We do not include our estimates of the aggregate impacts of evidence-based programs on forecasted prison populations or statewide crime rates. We proceed in two steps. The first step addresses the question: What works? Specifically, do rigorous evaluations indicate that some adult corrections programs, juvenile corrections programs, or prevention programs lower crime rates? To answer this fundamental question, we employ a systematic review of the research and use meta-analytic procedures to evaluate the evidence. While the purpose of the first step is to determine if anything works to lower crime outcomes, in the second step we ask a follow-up question: Per dollar spent on a program, do the benefits of the program’s crime reduction exceed its costs? Since all programs cost money, this additional economic test seeks to 171 172 E. K. Drake, S. Aos, and M. G. Miller determine whether the amount of crime reduction justifies the program’s expenditures. A program may have demonstrated an ability to reduce crime but, if the program costs too much, it may not be a good investment—especially when compared with alternatives including incarceration. We describe the economic model we have developed to predict how much money is spent or saved in Washington when crime goes up or down. META-ANALYTICAL PROCEDURES To estimate the benefits and costs of different approaches to reduce and prevent crime, we conducted separate meta-analyses of the relationship between evaluated programs and crime. In this section, we describe our procedures for searching for, including, and coding studies—along with the statistical methods we used to estimate the weighted average effects of a program. Search Strategy We searched for all adult and juvenile corrections and prevention evaluation studies conducted since 1970 that are written in English. We used three primary means to identify and locate these studies: (a) we consult the study lists of other systematic and narrative reviews of the adult and juvenile corrections and prevention research literature; (b) we examine the citations in the individual evaluations; and (c) we conduct independent literature searches of research databases using search engines such as Google, Proquest, Ebsco, ERIC, and SAGE. We obtained and examined copies of all individual program evaluation studies we could locate using these search procedures. Many of these studies were published in peer-reviewed academic journals, while others were from government reports obtained from the agencies themselves. It was important to include non–peer reviewed studies, because it has been suggested that peer-reviewed publications may be biased to show positive program effects (Lipsey & Wilson, 2001). Therefore, our meta-analysis includes all available studies we could locate regardless of published source. Criteria for Inclusion and Exclusion of Studies Comparison group. The most important inclusion criterion in our systematic review of the literature was that an evaluation must have a control or comparison group. We did not include studies with a single-group, pre-post research design in order to avoid false inference on causality (Coalition for Evidence-Based Policy, 2003). Random assignment studies were preferred for inclusion in our review, but we also included nonrandomly assigned control groups. We only included quasiexperimental studies if sufficient information was provided to demonstrate reasonable comparability between the treatment and comparison groups on important pre-existing conditions such as age, gender, and prior criminal Public Policy Options history. Of the 545 individual studies in our review, about 4\% involved effects estimated from well-implemented random assignment studies. Participant sampling procedures. We did not include a study in our metaanalytic review if the treatment group was made up solely of program completers. We adopted this rule to avoid unobserved self-selection factors that distinguish a program completer from a program dropout; these unobserved factors are likely to significantly bias estimated treatment effects (Lipsey, 2003). Some comparison group studies of program completers, however, contained information on program dropouts in addition to a comparison group. In these situations, we included the study if sufficient information was provided to allow us to reconstruct an intent-to-treat group that included both completers and noncompleters, or if the demonstrated rate of program noncompletion was very small (e.g., under 10\%). In these cases, the study still needed to meet the other inclusion requirements listed here. Outcomes. A crime-related outcome had to be reported in the study to be included in our review. Some studies presented several types of crime-related outcomes. For example, studies frequently measured one or more of the following outcomes: total arrests, total convictions, felony arrests, misdemeanor arrests, violent arrests, and so on. In these situations, we coded the broadest crime outcome measure. Thus, most of the crime outcome measures that we coded are total arrests and total convictions. When a study reported both total arrests and total convictions, we calculated an effect size for each measure and then took a simple average of the two effect sizes. Some studies included two types of measures for the same outcome: a dichotomous outcome and a continuous (mean number) measure. In these situations, we coded an effect size for the dichotomous measure. Our rationale for this choice was that in small or relatively small sample studies, continuous measures of crime outcomes can be unduly influenced by a small number of outliers, while dichotomous measures can reduce this problem (Farrington & Loeber, 2000). Of course, if a study only presented a continuous measure, we coded the continuous measure. When a study presented outcomes with varying follow-up periods, we generally coded the effect size for the longest follow-up period. This allowed us to gain the most insight into the long-run benefits and costs of various treatments. Occasionally, we did not use the longest follow-up period if it was clear that a longer reported follow-up period adversely affected the attrition rate of the treatment and comparison group samples. Miscellaneous coding criteria. Our unit of analysis was an independent test of a treatment at a particular site. Some studies reported outcomes for multiple sites; we included each site as an independent observation if a unique and independent comparison group was also used at each site. Some studies presented two types of analyses: raw outcomes that were not adjusted for covariates such as age, gender, or criminal history; and those that 173 174 E. K. Drake, S. Aos, and M. G. Miller had been adjusted with multivariate statistical methods. In these situations, we coded the multivariate outcomes. Procedures for Calculating Effect Sizes Calculations for dichotomous and continuous outcomes. Effect sizes measure the degree to which a program has been shown to change an outcome for program participants relative to a comparison group. In order to be included in our review, a study had to provide the necessary information to calculate an effect size. Several methods can be used by meta-analysts to calculate effect sizes. We used the standardized mean difference effect size for continuous measures and the D-cox transformation as described in Sánchez-Meca, Chacón-Moscoso, and Marín-Martínez (2003, Equation 18) to approximate the mean difference effect size for dichotomous outcome variables. ⎛ P (1 − pc ) ⎞ dCox = ln ⎜ e ⎟ / 1.65 ⎝ Pc (1 − pe ) ⎠ (1) In Equation 1, dcox is the estimated effect size, which is derived by dividing the log odds ratio by the constant 1.65. Pe represents the percentage outcome for the experimental or treatment group and Pc is the percentage outcome for the control group. For continuous outcome measures, we used the standardized mean difference effect size statistic (Lipsey & Wilson, 2001, table B10, Equation 1). ESm = Me − Mc SDe2 + SDc2 2 (2) In the second equation, ESm is the estimated standardized mean effect size where Me is the mean outcome for the experimental group, Mc is the mean outcome for the control group, SDe is the standard deviation of the mean outcome for the experimental group, and SDc is the standard deviation of the mean outcome for the control group. Sometimes research studies reported the mean values needed to compute ESm in Equation 2, but they failed to report the standard deviations. Often, however, the research reported information about statistical tests or confidence intervals that could then allow the pooled standard deviation to be estimated. These procedures are further described in Lipsey and Wilson (2001). Some studies had very small sample sizes, which have been shown to upwardly bias effect sizes—especially when samples are less than 20. Therefore, we followed Hedges (1981) and Lipsey and Wilson (2001, Equation 3.22) and report the “Hedges correction factor,” which we used to adjust all mean difference Public Policy Options effect sizes (N is the total sample size of the combined treatment and comparison groups). 3 ⎤ ⎡ ESm × [ESm , or, dcox ] ′ = ⎢1 − 4 N − 9 ⎥⎦ ⎣ (3) Techniques Used to Combine the Evidence Once effect sizes were calculated for each program effect, the individual measures were summed to produce a weighted average effect size for a program area. We calculated the inverse variance weight for each program effect and these weights were used to compute the average. These calculations involved three steps. First, we calculated the standard error of each mean effect size. For continuous outcomes, the standard error, SEm, was computed with (Lipsey & Wilson, 2001, Equation 3.23) SEm = ne + nc ( ES’ m )2 + 2(ne + nc ) ne nc (4) In Equation 4, ne and nc are the number of participants in the experimental and control groups and ESm is from Equation 3. For dichotomous outcomes, the standard error, SEdcox, was computed with (Sánchez-Meca et al., 2003, Equation 19) ⎡ 1 1 1 1 ⎤ SEdCox = 0.367 ⎢ + + + ⎥ ⎣ O1 E O2 E O1C O2C ⎦ (5) In Equation 5, O1E and O1C represent the success frequencies of the experimental and control groups. O2E and O2C represent the failure frequencies of the experimental and control groups. The second step in calculating the average effect size for a program area was to compute the inverse variance weight, wm, for each mean effect size with (Lipsey & Wilson, 2001, Equation 3.24) wm = 1 2 SEm (6) The weighted mean effect size for a group of studies was then computed with (Lipsey & Wilson, 2001, p. 114) ES = ∑ (wmES ′m ) ∑ wm (7) 175 176 E. K. Drake, S. Aos, and M. G. Miller Finally, confidence intervals around this mean were computed by first calculating the standard error of the mean with (Lipsey & Wilson, 2001, p. 114) SEES = 1 ∑ wm (8) The lower, ESL, and upper, ESU, limits of the confidence interval were computed with (Lipsey & Wilson, 2001, p. 114) ESL = ES − z(1− a ) (SEES ) (9) ESU = ES + z(1− a ) (SEES ) (10) In Equations 9 and 10, z(1-α) is the critical value for the z-distribution. Techniques Used to Assess Heterogeneity Computing random effects weighted average effect sizes and confidence intervals. Once the weighted mean effect size was calculated, we tested for homogeneity. This provides a measure of the dispersion of the effect sizes around their mean and is given by (Lipsey & Wilson, 2001, p. 116) Q = (∑ w ES2 ) − (∑ wES)2 ∑w (11) The Q-test is distributed as a chi-square with k-1 degrees of freedom (where k is the number of effect sizes). When the p-value on the Q-test indicates significance at values of p less than or equal to .05, a random effects model was performed to calculate the weighted average effect size. This was accomplished by first calculating the random effects variance component, v (Lipsey & Wilson, 2001, p. 134). v= Q − (k − 1) w − ∑ (∑ wsq ∑ w) (12) This random variance factor was then added to the variance of each effect size and all inverse variance weights were recomputed, as were the other meta-analytic test statistics. Adjustments to Effect Sizes Methodological quality. Not all research is of equal quality and this greatly influences the confidence that can be placed in interpreting the policy-relevant Public Policy Options results of a study. Some studies are well-designed and implemented and the results can be reasonably viewed as causal effects. Other studies are not designed as well and less confidence can be placed in the causal interpretation of any reported differences. Studies with inferior research designs cannot completely control for sample selection bias or other unobserved threats to the validity of reported research results. This does not mean that results from these studies are of no value, but it does mean that less confidence can be placed in any cause-and-effect conclusions drawn from the results. To account for the differences in the quality of research designs, we used a 5-point scale as a way to adjust the raw effect sizes. The scale is based closely on the 5-point scale developed by researchers at the University of Maryland (Sherman et al., 1998, chap. 2). On the 5-point scale as interpreted by our institute, each study was rated with the following numerical ratings. A “5” was assigned to an evaluation with well-implemented random assignment of subjects to a treatment group and a control group that does not receive the treatment/program. A good random assignment study should also report how well the random assignment actually occurred by reporting values for pre-existing characteristics for the treatment and control groups. A “4” was assigned to a study that employed a rigorous quasiexperimental research design with a program and matched comparison group, controlling with statistical methods for self-selection bias that might otherwise influence outcomes. These quasiexperimental methods might have included estimates made with a convincing instrumental variables or regression discontinuity modeling approach or other techniques such as a Heckman self-selection model (Rhodes et al., 2001). A value of 4 might also be assigned to an experimental random assignment design that reported problems in implementation, perhaps because of significant attrition rates. A “3” indicated a nonexperimental evaluation where the program and comparison groups were reasonably well matched on pre-existing differences in key variables. There must be evidence presented in the evaluation that indicated few, if any, significant differences were observed in these salient pre-existing variables. Alternatively, if an evaluation employed sound multivariate statistical techniques to control for pre-existing differences, and if the analysis was successfully completed and reported, then a study with some differences in pre-existing variables could qualify as a level 3. A “2” involved a study with a program and matched comparison group where the two groups lacked comparability on pre-existing variables and no attempt was made to control for these differences in the stu ... Purchase answer to see full attachment