RD601 - Immediate Sanction Probation Pilot Program Evaluation - December 20, 2016


Executive Summary:
The Hawaii Opportunity Probation with Enforcement (HOPE) program was established in 2004 with the goal of enhancing public safety and improving compliance with the rules and conditions of probation among offenders being supervised in the community. HOPE targets higher risk probationers and requires that each violation of the conditions of supervision is met with a swift and certain, but mild, sanction. A rigorous evaluation of HOPE completed in 2009 found a significant reduction in technical violations (such as drug use and missed appointments), lower recidivism rates, fewer probation revocations, and reduced use of prison beds among HOPE participants compared to similar offenders supervised on regular probation. Interest in Hawaii’s swift-and-certain sanctions model spread. As of July 2015, there were swift-and-certain sanctions programs operating in at least 29 states across the country.

The 2010 General Assembly passed legislation which established the basic parameters for swift-and-certain sanctions programs in Virginia (§ 19.2-303.5). In May 2012, the General Assembly adopted budget language to extend the provisions of § 19.2-303.5 and to authorize the creation of up to four Immediate Sanction Probation Programs (Item 50 of Chapter 3 of the 2012 Acts of Assembly, Special Session I). This provision charged the Virginia Criminal Sentencing Commission with selecting the pilot sites, developing guidelines and procedures for the program, administering program activities, and evaluating the results. As no additional funding was appropriated for this purpose, the pilot project was implemented within existing agency budgets and local resources. The General Assembly has since extended the sunset date to July 1, 2017, which enabled the pilot sites to continue the program until the 2017 General Assembly has reviewed the Commission’s evaluation and determined whether to continue the program in the future.

Since the 2009 HOPE evaluation, a number of programs based on the HOPE model have been evaluated. Results of these studies have been mixed. A longer term evaluation of HOPE completed in 2016, as well as evaluations in Washington State, Arkansas, Michigan, and Kentucky found that the HOPE approach yielded positive results, such as lower recidivism rates and reduced use of incarceration. However, a recent large-scale evaluation of a four-site replication of the HOPE model, funded by the Bureau of Justice Assistance (BJA) and the National Institute of Justice (NIJ), did not produce similar results. According to this evaluation, there were no statistically significant differences, overall, between the HOPE and probation-as-usual groups in the likelihood of arrest, new conviction, or probation revocation. Similarly, an evaluation of a Delaware program based on the HOPE model found that the program was not successful in reducing substance use or new crimes among probationers.

The Commission designed Virginia's Immediate Sanction Probation Program based on the parameters established by the General Assembly's statutory and budgetary language and the key elements of the swift-and-certain sanctions model pioneered in Hawaii. Implementing Virginia's program with as much fidelity as possible to the swift-and-certain sanctions model provided the best opportunity to determine if the positive results observed in HOPE and other programs would emerge in Virginia. Thus, the Immediate Sanction Program targets offenders who are at risk for recidivating or failing probation. Working with the Secretary of Public Safety and Homeland Security and the Department of Corrections, the Commission identified four pilot sites (Henrico County, the City of Lynchburg, City of Harrisonburg/Rockingham County, and Arlington County), which became operational between November 2012 and January 2014. The Commission developed policies and procedures to provide a framework for the program, including eligibility criteria and a mechanism for expedited hearings for program violations. In each site, Commission staff organized and participated in multiple meetings to facilitate and support local implementation of the program.

As of October 1, 2016, 288 probationers across the four pilot sites had been placed into the Immediate Sanction Probation Program. In order to allow for a sufficient follow-up period to track participants for recidivism, the 200 eligible participants who were placed into the program before July 1, 2015, were selected for the evaluation cohort. The majority (76%) were at medium to high risk of recidivating and all had a history of technical violations prior to program placement. Low risk probationers were only placed in the program after committing at least three violations while on regular supervision, indicating a higher risk for revocation. More than 80% of participants violated at least once after program placement, committing an average of 2.7 violations each. The most common violation during program participation was drug use. As of October 1, 2016, 39% of the evaluation cohort had completed the program. Nearly all of the program completers had been violation-free for 12 months, the measure established by the Commission for “successful completion.” Judges allowed seven participants who had not reached the 12-month violation-free mark to complete the program, due to individual circumstances of these participants.

The Commission used standards established in the 2016 evaluation of the BJA/NIJ-funded HOPE replication project to measure the swiftness and certainty of sanctions imposed during Virginia’s pilot program. For swiftness, pilot sites were assessed based on the percentage of violations heard by the court within three days. Approximately half (47%) of program violations in Virginia’s pilot sites were heard by the court within the three-day window. This is below the minimum of 60% established by the evaluators of the HOPE replication project. Regarding the certainty of sanctions, Immediate Sanction judges responded to violations by imposing a jail sanction for 100% of the violations brought to court, per the program’s design. Judges utilized jail sanctions as envisioned by the Commission, with more than 94% of sanctions falling within the recommended range. Nearly 93% of the jail sanctions imposed were at or below the maximum sanction of 19 days used by evaluators of the HOPE replication project.

The Commission tracked the evaluation cohort for one year following placement into the Immediate Sanction Program. At the one-year mark, 9.7% of the participants in the evaluation cohort had been arrested for a new felony. Only 6.2% had a new felony conviction based on an offense committed during the follow-up period. Participants whose primary drug of use was opiates (including heroin) recidivated at a higher rate than other participants.

For the evaluation, the Commission developed a quasi-experimental design, often used in evaluations of criminal justice programs. Quasi-experimental designs identify a comparison group that is as similar as possible to the program or treatment group in terms of baseline (pre-intervention) characteristics. To reduce the risk of bias (i.e., the possibility that participants are systematically different from nonparticipants), the Commission used commonly accepted statistical techniques to create a valid comparison group. Constructing the comparison group for this evaluation was a two-stage process. In the first stage, the Commission identified jurisdictions that were similar to the pilot sites across a number of community-level characteristics, such as crime rates, demographics, and judicial practices in sanctioning technical probation violators. In the second stage, the Commission developed a pool of potential comparison offenders from within the selected comparison jurisdictions. Using tightly controlled matching procedures, the final sample included 63 participants in the evaluation cohort matched to 63 comparison probationers, for a total of 126 subjects. Participants for whom no matched comparison probationer could be found were not included in the subsequent analyses.

At one year from program placement or, in the case of the comparison group, one year from the date the probationer would have become eligible for placement, 7.9% of the 63 participants in the matched sample had been rearrested for a felony offense versus 22.2% of the comparison group. Thus, Immediate Sanction participants were less likely than comparison probationers to be rearrested for a felony during the one-year follow-up. Immediate Sanction participants were also less likely than comparison probationers to be reconvicted of a felony following the arrest (6.3% for participants versus 17.5% for the comparison group). The Commission conducted survival analysis, which measures the time until a recidivist event occurs, to determine if these differences were statistically significant. The results of the survival analysis are mixed. This analysis revealed that Immediate Sanction participants were less likely to be rearrested for a felony over time than those in the comparison group and were free of felony arrests for a longer period of time. When controlling for relevant factors, including street time (i.e., the time that the individual was not in jail serving sanctions, etc., and, thus, was in the community with the opportunity to recidivate), this finding remained statistically significant (p<.05). However, when examining the time until rearrest for an offense that resulted in a felony conviction, the differences between participants and the comparison group were not statistically significant after controlling for other factors. Due to the small sample size and relatively low occurrence of recidivism, the results of the Commission’s analyses are not generalizable to the population.

The Commission also compared probation revocation rates. Immediate Sanction participants in the matched evaluation cohort had their probation revoked at a higher rate than comparison offenders (30.2% compared to 23.8%). Not only were participants more likely to be revoked, sentences imposed for revocations were more severe for program participants. Immediate Sanction participants in the matched evaluation cohort were much more likely to receive a prison term when revoked than offenders in the comparison group. The stark differences in the outcomes for revocations had a considerable impact on the cost analysis, which revealed that the Immediate Sanction Probation Program in Virginia costs more than traditional probation. This finding suggests that differences in probation and judicial practices or differences in the amenability of certain offender populations may influence the outcomes and costs of implementing programs based on the HOPE model (Lattimore et al., 2016a).

Implementing and maintaining a new program that diverges substantially from existing practice is very challenging. Over the course of the pilot program, the Commission noted a number of successes but also some difficulties that were not entirely overcome. Successes are listed below.

• Ensuring that violations are addressed immediately and cases are handled swiftly requires extensive collaboration and coordination among many criminal justice agencies and offices. The majority of stakeholders in the pilot sites demonstrated a commitment to working with each other, and the Commission, to give the pilot program the best opportunity to succeed despite the lack of funding.

• Local stakeholders executed an entirely new structure for handling violations in an expedited fashion, where such a process had not existed previously.

• Virginia’s Immediate Sanction Probation pilot program achieved a number of the key targets of the HOPE model:

* The majority of the individuals placed in the program had scored medium to high on the COMPAS recidivism risk scale. Low risk probationers placed in the program had committed numerous violations while on regular supervision, placing them at higher risk for revocation.

* For participants who violated program rules, 100% of the violations resulted in a jail sanction and these sanctions were served immediately following the hearing. Nearly all were within the range recommended by the Commission and were at or below the maximum sanction established in the evaluation of the BJA/NIJ-funded HOPE replication project.

• The vast majority of participants who completed the program were released from any remaining supervised probation obligation. Release from supervision for successful completers serves as an incentive for participants in the program.

The successes of the pilot sites should not overlooked. They are a testament to the dedication and extensive collaboration of the more than 100 stakeholders in the local pilot sites. A number of challenges remained, however, and some key targets were not reached. These are described below.

• The number of program candidates referred by probation staff was lower than expected. This may be due, in part, to the eligibility criteria, which excluded those with obligations to courts outside of the pilot jurisdictions and, by statute, those on probation for an offense defined as violent in § 17.1-805. Also, the Commission had no ability to ensure that all eligible probationers in the pilot sites were referred to the program. The lower-than-expected number of referrals posed a challenge for the evaluation.

• Other evaluators have found that strong local leadership is very important to the successful implementation and continued fidelity to the HOPE model. While the Commission met with local stakeholders regularly from program implementation through June 2015 to provide guidance and assistance in addressing obstacles, the pilot program likely would have benefitted from the leadership of a highly-involved local stakeholder serving as a champion in each of the pilot sites. While not possible given the Commission’s budget constraints, having an on-site project coordinator in each location, such as that provided to sites participating in the BJA/NIJ-funded HOPE replication, would have been beneficial for program fidelity and data collection.

• Arguably the most significant difference between Virginia’s Immediate Sanction Probation Program and Hawaii’s HOPE program (and the recent four-site replication of HOPE) has been the lack of resources for substance abuse services, particularly residential/inpatient options. Moreover, the availability of treatment resources varies considerably across jurisdictions in Virginia and, in at least one pilot site, very few treatment services are available for probationers under supervision in the community.

• Limited staff resources presented additional challenges at times. The intense supervision of new participants, in conjunction with immediate arrests, hearings, and jail time for violations, meant that existing resources were stretched thin. Relatively high turnover, particularly in the probation offices, at times made it difficult to maintain an experienced corps of program personnel.

• The Commission observed some inconsistencies across the pilot sites in the supervision practices of Immediate Sanction probation officers, for example, the extent to which the results of handheld urinalyses were sent to the centralized laboratory for confirmation prior to effecting an arrest.

• Similar to the findings of the HOPE replication project, Virginia’s pilot sites had difficulty bringing participants to a violation hearing within three days, in large part because of participants who failed to show up at scheduled appointments or who absconded. Although the pilot sites were successful in implementing a much faster process to bring a violation before the court, none of the sites achieved the minimum 60% target for the percentage of violations handled within three days recommended by the evaluators of the HOPE replication project. Examining the data further revealed that while roughly half (47.3%) of violation hearings occurred within three days of the violation, the vast majority (92.5%) of hearings were held within three business days following arrest. This ranged from 84.7% of violations in Lynchburg to 100% in Henrico. Whether the stakeholders had selected set days and times to conduct these hearings (e.g., every Monday, Wednesday, and Friday at 1:00 p.m.) as well as judicial caseload and other factors may have played a role in this variation across pilot sites.

Maintaining fidelity to the model over the long term is particularly challenging for a program of this kind. Given that the Immediate Sanction Program is a significant departure from current practice and the need for buy-in from a large number of stakeholders, strict adherence to program protocols may be difficult to maintain over time. Current policies of the Department of Corrections allow for discretion of the probation officer in the supervision of his or her caseload and, in some respects, encourage Virginia’s 43 probation districts to develop localized practices. A program like HOPE, which requires strict adherence to uniform protocols and removes discretion from the officer in the handling of violations, may be difficult to implement and sustain in the context of traditional probation in Virginia.

While evaluations of Hawaii’s HOPE program and others have found lower recidivism rates and reduced use of incarceration for probationers, the Commission’s analysis of Virginia’s pilot program yielded mixed results. The analysis suggested that Immediate Sanction participants were more likely to be free of felony arrests for a longer period of time than comparison offenders (p<.05). While Immediate Sanction participants were also less likely than comparison probationers to be reconvicted for a new felony, the relationship between program participation and subsequent felony reconviction was not statistically significant after controlling for other factors. Moreover, participants in the Immediate Sanction Program were more likely to have their probation revoked than comparison probationers and, when revoked, were much more likely to receive a prison term than those in the comparison group. Cost analysis indicated that the Immediate Sanction Probation Program costs more than traditional probation in Virginia. The Commission’s evaluation is limited by the small sample size and the relatively low occurrence of recidivism; therefore, the results are not generalizable to the population. Data limitations, most notably for individuals on traditional probation, meant that certain aspects of the program, such as utilization of treatment services, could not be included in the recidivism or cost analysis.

At the close of 2016, a growing number of swift-and-certain sanctions programs have been evaluated and, while this has greatly contributed to the body of research on this model of community supervision, mixed results have emerged. Several studies found positive program effects, while at least two recent studies (including the large-scale HOPE replication project) did not. Additional research is needed to determine why some swift-and-certain sanctions programs are effective at lowering recidivism and reducing the use of incarceration and others are not and, in particular, for which offender populations this approach is most effective.