The IPS Learning Community: A Longitudinal Study of Sustainment, Quality, and Outcome

The American Psychiatric Association has published the results of a longitudinal survey into the provision of IPS services in the USA.

Objective:

Implementations of evidence-based mental health practices often disappear quickly, and few studies have examined sustainment. Since 2001, the Individual Placement and Support (IPS) learning community has promoted dissemination, implementation, sustainment, and expansion of IPS by using multiple strategies: online training, in-person training and technical assistance, technical assistance teleconferences, annual meetings, stakeholder conference calls, fidelity assessments, and transparency of outcomes. This study examined sustainment of IPS over a two-year period among programs in the learning community in the United States.

Methods:

The authors interviewed IPS team leaders in 129 programs actively participating in the learning community in 2012 and 2014. The structured interview addressed questions about program status, funding, and quality improvement activities. Simultaneously, the learning community tracked program-level data on IPS fidelity and employment rates. The study examined two-year program sustainment and changes in employment rates, fidelity, funding, and quality improvement activities.

Results:

In 2012, 129 participating sites had been active for an average of 4.5 years. At two-year follow-up, 124 (96%) sites were sustained. The sustaining sites maintained quality improvement activities, expanded funding sources, and increased employment rates (41% to 43%; p=.04) and fidelity scores (103.8 to 108.4; p=.002).

Conclusions:

Nearly all programs participating in a learning community in 2012 continued to provide IPS services over the next two years, exceeding sustainment rates for evidence-based practices reported in the literature. Quality indicators also improved, suggesting that learning community activities fostered sustainment and quality. Controlled studies must compare specific learning community approaches with usual methods of sustainment.

Even though many evidence-based practices have been developed, few clients with severe mental illness receive effective services. Evidence-based practices are often implemented poorly and rarely endure beyond initial enthusiasm and grant funding. The empirical literature on sustainment of evidence-based practices is weak and fragmented. Factors commonly hypothesized to influence sustainment include political support, funding stability, community partnerships, organizational capacity, fidelity monitoring, technical assistance, workforce development, and supervision. Although many individual factors have modest empirical support, research findings have been inconsistent.

Monitoring the quality of services is crucial; without mechanisms to ensure adherence to model standards, programs will vary widely in how services are delivered. Therefore, fidelity scales, defined as measures that assess adherence to a program model, have become essential tools for program implementation and sustainment. Strategies to maintain and improve the quality of services include establishment of learning collaboratives and technical assistance centers to provide training, resource materials, fidelity monitoring, on-site consultation, and other sources of support. At the local agency level, quality improvement efforts, including fidelity and outcome monitoring, are associated with better implementation. The frequency and quality of supervision also affect the quality of implementation and sustainability of services. Another factor, workforce development, ensures that practitioners have the skills to do the practice. At the state level, ensuring a well-trained workforce requires systematic and large-scale methods for initial and booster training; online training is one such method.

One evidence-based practice increasingly implemented throughout the United States is the Individual Placement and Support (IPS) model of supported employment. In recognition of the many factors influencing the long-term survival of any program model, a comprehensive learning community was developed to sustain IPS. Beginning in 2001, the Dartmouth Psychiatric Research Center and the Johnson & Johnson Office of Corporate Contributions partnered to develop a comprehensive program to strengthen state and local infrastructures to promote access to IPS throughout the United States. After starting as a small demonstration in three states, the program has evolved into a network of 20 states and three European countries known as the IPS learning community.

Historically, the term “learning collaborative” has been used to define a network of organizations with a shared goal of improving treatment for a specific medical condition, facilitated by regular communication (for example, meetings, teleconferences, and newsletters) and collection and dissemination of objective information about procedures and outcomes, typically over a few months. Some collaboratives provide training and technical assistance and facilitate research and innovation. The IPS group adopted the term “learning community” to signify its long-term commitment to quality and expansion; the term differentiates our approach from time-limited quality improvement learning collaboratives.

The purpose of this descriptive study was to examine sustainment of U.S. programs in the IPS learning community over a two-year period. We also examined changes over that time in the infrastructure supporting sustainment. We hypothesized that participation in the learning community would promote sustainment, program fidelity, and employment outcomes.