HD3 - Review of the ADAPT System at the Department of Social Services


Executive Summary:
The Application Benefit Delivery Automation Project (ADAPT) is a computer systems project designed to automate the eligibility determination process for three major social services benefit programs -- Aid to Families with Dependent Children (AFDC), Food Stamp, and Medicaid. The Department of Social Services (DSS) initiated the project to help local social service agencies process client applications for benefit programs more efficiently and effectively.

The ADAPT project began in 1991, with completion originally scheduled for March 1993. However, in 1992 with the concurrence of the local social services agencies, DSS modified the scope of the project significantly to incorporate a rules-based design. A rules-based system takes client information provided by the eligibility worker and makes eligibility determinations and benefit calculations using a programmed set of rules.

In December 1995, the Secretary of Health and Human Resources directed the Commissioner of Social Services to freeze the development and further implementation of the ADAPT project. The commissioner recommended that the implementation and training for ADAPT be halted until completion of: (1) an assessment of ADAPT by a systems integrator, (2) a study of the capacity requirements to process the Food Stamp component of the system, and (3) an analysis of federal block grant changes and welfare reform as it applies to the ADAPT project.

Although the suspension seems understandable, the decision to suspend the project came almost one full year after DSS had received critical information about the implementation of ADAPT from its own consultant. In the meantime, the project team, which was not told of concerns about project performance, continued to move forward with implementation of the Food Stamp component of the ADAPT system, and DSS incurred about $6.7 million in ADAPT-related expenses. The critical consultant report and subsequent information about problems with the project were not made available to the General Assembly, federal government agencies, and local social service agencies until after the project was suspended. Consequently, in light of on-going development of the project and the significant expenditures during calendar year 1995, the decision to suspend the project was unexpected and surprising to most of those involved in the system’s development.

Item 15 of House Bill 29 (1996) directed the Joint Legislative Audit and Review Commission (JLARC) to investigate the Department of Social Services’ procurement and implementation of the ADAPT project. The 1996 General Assembly directed this review because of conflicting information about the functionality of the system from DSS and the 10 localities in the first phase of implementation using the system. The study mandate directed JLARC to report its findings by June 30, 1996, to the chairmen of the following committees: House Appropriations; House Health, Welfare and Institutions; Senate Finance; and Senate Rehabilitation and Social Services.

This review of the implementation of the ADAPT system and procurements for the project found that:

• though innovative, the inclusion of the rules-based design added greatly to the complexity of the ADAPT project and resulted in significant delays in completing the project;

• DSS has spent about $20.2 million for the ADAPT project, most of which was for the development of the rules-based system; it appears that DSS may have very little money available to complete the development of the system without additional appropriations and federal government approval; Chart: ADAPT Project Expenditures by Year

• successful project implementation has been impeded by fragmented authority and responsibility, poor financial management, and the lack of continuous high level management support of the project;

• despite the incomplete status of the ADAPT system, the Food Stamp portion of the project is currently operational in 10 localities; it provides these localities with needed automated tools to assist eligibility workers and is easy to use;

• many of the technical concerns which led to the suspension of the project appear to have been at least partially addressed; and

• DSS is now considering alternatives to ADAPT which will require significant new development and additional funding; however, the department has failed to involve its local partners in the decisionmaking process to determine the future of the ADAPT system.

While the current system may not be the ideal system, it appears to be a workable solution and should be considered equally with other alternatives. Yet, it does not appear that DSS has fully assessed the viability of the current system to provide the functionality required by local social service agencies or its attendant costs. In making decisions about automation of the eligibility determination process, DSS needs to weigh the trade-offs between time, flexibility, and costs. A formalized methodology is needed to assess those trade-offs to ensure a non-biased, critical assessment of the alternatives. In particular, DSS needs to provide evidence that the alternatives being proposed provide tangible benefits that outweigh the time delays, complexity, and potentially higher cost of implementing other alternatives.

The department also needs to include others in the decisionmaking process for ADAPT. The DSS technical and program staff, local social service agencies, and several other State agencies should help to assess the future of the system. High level support and leadership is needed to rebuild the State/local partnership to complete the implementation of ADAPT. Without a partnership of State and local agencies, automation of local social service programs cannot succeed. As part of this effort, the Secretary of Health and Human Resources and the DSS commissioner need to make completion of this project a high priority.

The Rules-Based Design of ADAPT Was Innovative

The inclusion of a rules-based system design in the ADAPT project was an innovative, bold initiative for DSS. It demonstrated a commitment from the State to respond to the needs of local social service agencies. Further, its development served as a model for building State and local partnerships for systems development projects. Although it encountered many obstacles in the planning, development, and implementation of ADAPT, the department achieved several significant successes. Most notably, the ADAPT project team completed several of the initial goals established for the project, such as:

• building a "bridge" between the Medicaid Management Information System (MMIS) and the Virginia Client Information System (VACIS) for the purposes of enrolling Medicaid eligible individuals in the Medicaid program,

• installing enhancements to VACIS,

• enhancing parts of the benefit application process, including completing the Food Stamp component of the rules-based system and completing more than 90 percent of the AFDC component of the rules-based system, and

• completing a planning document for the long-term direction of social services information systems.

Further, the involvement of the local social service agencies in workgroups to plan for the implementation of the system, and to help develop and test the rules base was, in the words of one local official, ". . . the most creative sharing with local governments that the State has ever tried to do."

Implementation of the Rules-Based System Was Problematic

The biggest obstacle to the successful implementation of ADAPT has been the completion of the rules-based system for automating eligibility determination and benefit calculation, especially for the Medicaid program. The rules-based design added to the complexity of the ADAPT project, ultimately leading to several modifications to the project schedule when it became clear that DSS could not meet the aggressive schedules set for it. The implementation of the rules-based system was hampered by several factors, such as:

• the transfer of the Napa County system (NAPAS) for use in Virginia was more complicated than originally expected;

• DSS management had an unrealistic perception of the resource requirements needed for the project, leading to a constant struggle by the project team for adequate resources to complete the project requirements; and

• problems related to the development of the Medicaid program rules.

Problems with the implementation of the project resulted in changes to the project’s schedule, which initially called for full implementation of the system in early 1993. Ultimately, the project was to begin statewide implementation in September 1995. Even this modified schedule could not be met, however.

The ADAPT Project Has Been Impeded by Poor Management from Its Beginning

Implementation of the ADAPT system was also impeded by problems related to project management. Authority and responsibility for the project was diffused and fragmented. This contributed to poor financial management, procurement practices, and inventory control of computer equipment purchases. For example, JLARC found that computer equipment valued as much as $2 million was inaccurately inventoried. Until the department completes a full accounting of this equipment, it cannot ensure that this equipment has not been lost, stolen, or inappropriately assigned.

Furthermore, high level support for the project was lost with the change in administrations. Beginning in 1994, new DSS organizational priorities replaced the priority status of ADAPT. This made it even more difficult for the project to sustain a sense of urgency, maintain adequate resources, and obtain additional resources for its completion.

Suspension of the ADAPT Project Has Had Negative Consequences

Further development and implementation of ADAPT is currently on hold pending the results of a DSS-commissioned management and systems engineering review of the project. The decision to suspend the ADAPT project appears to have been made with insufficient consideration of the serious ramifications this would have for the Commonwealth, and was implemented without sufficient foresight and planning. Moreover, DSS did not build the necessary support for the suspension by involving local agencies in the decision process. Thus, to those not involved in the decision, the suspension seemed to be a sudden, unexplained shift in direction for a long-standing project.

It is not surprising then, that the suspension of the ADAPT project has had a significant impact on the administration of social services programs at the local level. The local social services agencies re-engineered their eligibility processes at great cost and effort with the expectation that the State would provide the necessary automated tools to make the process work. Now, without the ADAPT project, the local agencies find themselves with significant workload, and little support or guidance from DSS on how to proceed. By suspending the project just as some local agencies were finally beginning to use the system and benefit from it, DSS has seriously eroded the confidence of the localities.

DSS’ failure to adequately involve the local agencies in decisions related to the suspension of ADAPT has negatively affected the State and local agency partnerships created through the ADAPT project. The decision may also have jeopardized:

• federal financial participation in the funding provided to date on the ADAPT project and any future funding for re-engineering proposals,

• local agency efforts to improve application processing times for the Food Stamp program and to process benefits more accurately,

• General Assembly support for future agency automation projects, and

• public confidence in the agency’s ability to deliver cost effective information technology solutions.

System Improvements Have Addressed Some Technical Concerns

Concerns were raised about the project’s original design and its ability to meet its stated objectives by a DSS consultant in January 1995. The issues raised were perceived as grave enough to provide some basis for the Secretary of Health and Human Resources to direct the DSS commissioner to suspend the project. For more than a year, DSS considered the advice of consultants who raised serious concerns about the usefulness and cost effectiveness of the system.

However, the actual viability of the current system has never been fully assessed, particularly since significant systems improvements have been made. Some of the concerns raised by DSS’ consultants were, in fact, irrelevant to the project delays and concerns about costs. Other issues, especially those related to hardware capacity and systems design, have been partly addressed by several system improvements. In deciding to suspend the project, DSS did not consider useful information from the local social services agencies, its own technical staff, or other State agencies involved in the project.

The ADAPT system, as it is currently implemented, is a working system. Although the technology deployed has certain limitations, some of those limitations have been overcome. For example, the ADAPT project team has created indexes and keys to improve database implementation, designed utilities to ease maintenance of the system, and developed methods to extract data for ad hoc management reports.

Moreover, certain portions of the system are now operational. In November 1995, DSS began statewide implementation of the Food Stamp component of the ADAPT system. At that time, the AFDC component of the project was more than 90 percent complete, and the eligibility determination and benefit calculation portion of the rules-based system had been successfully tested. In addition, a portion of the Medicaid requirements had been written and were awaiting completion of statewide conversion for the Food Stamp and AFDC programs before coding would resume.

Future Implications for the Full Implementation of ADAPT

Completion of ADAPT will be a significant effort, requiring support from both the executive branch and the General Assembly. It will also require a substantial commitment of additional funds, estimated to be between $3.5 million and $16.5 million. The additional funding needed to complete ADAPT’s development is problematic, however. The 1996-1998 biennial budget did not include any funds to support future development and implementation of ADAPT, though funding for processing in the 10 pilot and phase one localities is included. Therefore, it is unclear how the current re-engineering efforts conducted by DSS for ADAPT will be funded in the short term. Assuming that ADAPT funding will be revisited during the 1997 General Assembly Session, it is unlikely that additional funds will be available for project completion until at least July 1, 1997.

While the current implementation of ADAPT may not be the technically ideal solution, it does appear to be a workable solution and should be considered equally with other alternatives. The preliminary re-engineering analysis performed for DSS is not sufficiently detailed to determine whether it is either technically necessary or cost effective to seek an alternative to the current system. As a next step, DSS needs to more fully evaluate the current system. It needs to judge the appropriateness of a systems re-design for ADAPT only after it has carefully evaluated a full range of alternatives using established criteria. The evaluation process should involve all of the agencies -- State, local, and federal -- that were a part of the initial ADAPT partnership.

In assessing the current implementation of ADAPT against possible alternatives, decisionmakers need to consider the following questions:

• What are the total life-cycle costs associated with each alternative? If one of the alternatives proposed is more cost effective, does the reduction in costs outweigh the implications of delayed implementation of the system?

• Are end users willing to accept any shortcomings of the current implementation? Are the local social service agencies able to deal with further time delays that would be necessary to implement an alternative technology?

• What is gained by selecting an alternative implementation in terms of service quality? Does the potential improvement in service quality outweigh the implications of delayed implementation of the system?

• Will DSS limit itself in the future by implementing ADAPT as a MAPPER system? Will another technology also have this limiting effect?

In answering these questions, decisionmakers need to weigh the viability of the current system against proposed alternatives, using a systematic approach involving all the stakeholders. The problems experienced with the ADAPT project are not unusual ones for large system development projects. However, these problems have illustrated some significant weaknesses that will have to be addressed by DSS in order to successfully implement ADAPT. A high level of commitment from the executive and legislative branches, coupled with the involvement of local partners, is needed for the eventual successful completion of the project.