For the last few decades, community-based coalitions have been used as a strategy to reduce the prevalence of substance abuse disorder (Butterfoss, 2007; Roussos & Fawcett, 2000). In fiscal year 2019 alone, the federal government invested approximately $24 million dollars to support such coalitions via the Drug Free Community (DFC) Program (Office of National Drug Control Policy, 2019). While there is evidence of the effectiveness of the DFC Program overall (ICF International, 2017, 2019), there is considerable variability among individual coalitions in their abilities to reduce the prevalence of local substance use and to sustain themselves after federal funding has ended (e.g., Scheirer, 2005). Consequently, considerable support has also been directed to technical assistance organizations at the state, regional and national levels (Anderson-Carpenter et al., 2017; Watson-Thompson et al., 2013) to bolster coalition efficiency and viability.

This brief report describes the efforts of one technical support organization in Connecticut—the Prevention Training and Technical Assistance Service Center (PTTASC)—to create a coalition vitality assessment tool using participatory research methods and presents some preliminary observations from its initial use.

Context

In 2015, the Connecticut Department of Mental Health and Addiction Services (DMHAS) received federal funding to support local coalitions addressing substance abuse disorders and to provide technical assistance in these efforts. DMHAS contracted with Cross Sector Consulting, LLP, the lead partner of the PTTASC project, to support coalitions in implementing the Substance Abuse & Mental Health Administration Strategic Plan Framework (SPF) five step planning process.

Two of the authors’ past experience and research (Goldstein et al., 2017) has documented that a disproportionate amount of the leadership and administrative work of many coalitions falls upon the coalition coordinator, typically the only paid individual in the coalition (see also Doll et al., 2012). The coalition coordinator role was originally conceived (Evensen & Weatherly, 2009) of as providing administrative and logistical support to the leadership provided by the volunteer coalition members. In practice, many coalition coordinators become the de facto leaders of the coalition and are often expected to provide almost all of the leadership and logistic functions, which underutilizes the human and social capital contained within the coalition. Given this dynamic, many coalitions operate at levels analogous to stage 2 or 3 of Tufte and Mefalopulos’s (2009, pp. 6–7) four stage typology of participation: “participation by consultation” or “participation by collaboration.” The former describes participation as ". . . an extractive process, whereby stakeholders provide answers to questions posed by outside researchers or experts (coalition coordinator) . . . but “this consultative process keeps all the decision-making power in the hands of external professionals” (p. 6). Participation by collaboration involves “stakeholders participat[ing] in the discussion and analysis of predetermined objectives . . . [which] does not usually result in dramatic changes in what should be accomplished . . .” (p. 6-7).

Recognizing that maximal utilization of human and social capital would occur under conditions of "empowerment participation [“where primary stakeholders are capable and willing to initiate the process and take part in the analysis” (Tufte & Mefalopulos, 2009, p. 7)], PTTASC decided to engage the coalitions in a participatory research process of self-assessment to expand their understanding of their roles as coalition members and the capacities needed for the coalition to be successful. The first step was to create an instrument–the Coalition Vitality Assessment Tool (CVAT)–to help coalitions examine their strengths and weaknesses and identify specific capacity needs; this was followed by action steps to address the identified capacity needs.

Using participatory processes to develop the CVAT

After creating a preliminary draft for the CVAT, PTTASC next worked with the Evidence-Based Work Group (EBWG), a volunteer work group of evaluation specialists who consult with the state regarding evidence-based prevention programs and practices. After several meetings of with EBWG feedback, PTTASC produced a test version of the CVAT that incorporated coalition effectiveness research from both academic and practitioner perspectives.

Israel, Schulz, Parker, Becker, Allen and Guzman (2018) identified eight principles of community-based participatory research (CBPR) which we used to classify the various activities that PTTASC, the EBWG, and the coalitions engaged in during the development and application of the CVAT (see Table 1).

Table 1.Principles, descriptions, and exemplars of CPBR as manifested in our work with the CVAT and the coalitions.
Principle Description Exemplars from CVAT development and/or application
Recognize community as unit of identity Individual and collective identification based on geography and/or shared socially constructed values. The CVAT was refined through interactions among two different communities: the EBWG and multiple test-site coalitions representing distinct communities across the state.
Build on strengths and resources of the community “Identify and build on strengths, resources and relationships that exist within communities of identity . . . to support or expand social structures and social processes that [help] community members to work together to improve health” (Israel et al, 2018, p. 49-50). PTTASC asked coalition leaders and members to self-assess their coalition’s strengths and weaknesses using the CVAT and, following the discussion of all responses, to identify a minimum of three topic areas for capacity improvement.
Facilitate collaboration “All parties participate in and share control over all phases of the research process . . .” (Israel et al, 2018, p. 50). PTTASC meetings with coalition members were conducted in ways that shared power with coalition participants. PTTASC facilitated the process and coalition staff and members identified priorities for focused improvement.
Integrate knowledge and action for mutual benefit Information from participants is translated into action steps to address concerns of the community. CVAT feedback came from pilot testing with coalitions as well as EBWG members. Information from both the academic literature and personal knowledge of the community were used in the development of the CVAT and the cooperative action planning that followed.
Promote co-learning that attends to social inequities “Facilitates the reciprocal transfer of knowledge, skills, capacity and power . . . recognizing that . . . marginalized communities often have not had the power to define their own experience” (Israel et al, 2018, p. 50). During pilot testing, several coalitions identified active youth involvement as a missing factor in our capacity measure. Several items addressing youth capacity were subsequently added to the tool.
Address health from both positive and ecological perspectives Approaches health from a non-pathologizing perspective that emphasizes physical, mental and social well-being and considers an ecological perspective that includes biomedical, social, economic, cultural and political factors as determinants of health. While as a technical assistance organization PTTASC did not provide direct services to individual communities, it encouraged coalitions to explore the sociocultural factors unique to each community that contributed to substance abuse.
Involves a cyclical and iterative process A recursive process that includes partnership development and maintenance and all stages in the research process from problem identification to data interpretation, dissemination of results and action taking. The development of the CVAT involved multiple rounds of discussion, revision and refinement among EBWG members, the public, and test-site coalitions to create the working version of the instrument. A similar iterative process occurred when coalitions completed, scored and interpreted the data and made action plans for change. These steps were supported by PTTASC, which developed a technical assistance plan to support action plan implementation.
Disseminate findings to all partners. “ . . .disseminate findings and knowledge gained to all partners involved, in language that is understandable and respectful . . .” (Israel et al, 2018, p. 51). CVAT results are normally distributed to all coalition members for feedback, increasing ownership in the resulting action plan for coalition development.

The instrument is a broad-based grid which asked coalitions to assess their actions in four broad domains of coalition functioning: (1) coalition characteristics, (2) leadership, (3) action planning, and (4) sustainability. Each domain is divided into a number of elements which are further divided into a number of specific dimensions (See Table 2 for a listing of domains and elements; dimensions are not shown for proprietary reasons). Each dimension is then rated on a four-point scale by the coalition coordinator and a group of coalition members. Scores are then summed for each domain. We have illustrated one element and its rating scale in Table 3.

Table 2.Domains, elements and dimensions of the Coalition Vitality Assessment Scale.
Domain 1. Coalition composition
Key elements
Guiding principles
Coalition composition
Coalition meetings (full group)
Committees, work groups, or action teams
Culture
Domain 2. Leadership
Key elements
Coalition chair(s)
Coalition leaders (includes committee/work group leaders; NOT coalition staff)
Leadership development and succession
Coalition coordinator
Administrative support
Domain 3. Action Plans
Key elements
Fundamental knowledge of process
Data access and use
Coalition process to develop action plan
Action plan implementation process
Domain 4. Sustainability
Key elements
Fundamental knowledge of sustainability
Current plan
Process for financial reporting, resource development & sustainability
Table 3.An illustration of the Vision & Mission element and scoring criteria within Domain 1, Coalition Characteristics.
Domain 1. Coalition Characteristics 1 point

Limited to no capacity 		2 points

Concerns or issues limit coalition effectiveness 		3 points

Sufficient capacity or capabilities to produce results 		4 points

Strong capacity or capabilities

Self-Assess-ment Score
Guiding Principles Vision & Mission No vision or mission statement exists. Members ad lib the vision and mission. An outdated vision or mission statement exists. A current vision or mission statement exists and 51% to 75% of members can articulate the vision and mission. A current vision or mission statement exists and at least 76% of members can articulate the vision and mission.

Applying CVAT with the coalitions

Table 4 summarizes the activity stages involved in the application process; the timeframe for the first four activities is approximately sixty days, while the Document activity is ongoing.

Table 4.Activity sequence and description of the CVAT application process with coalitions.
Activity Description
Communicate
  • PTTASC staff describe value of CVAT process to coalition coordinators
  • Provides documents
  • Works with coalition coordinators to set up local CVAT process and timeline
Assess
  • Coalition assembles self-assessment team which completes self-assessment
  • Self-assessment team produces summary scorecard, sends it to PTTASC, and schedules follow-up session.
Design
  • PTTASC coach leads reflection process with self-assessment team to help identify top 3 to 5 improvement priorities
  • Self-assessment team and/or coalition confirms priorities
  • PTTASC coach helps coordinator develop one page work plan
Do
  • Coalition members and coalition coordinator work the plan
  • PTTASC provides ongoing coaching and technical assistance
  • Coalition coordinator and members attend PTTASC trainings
Document
  • Monthly check-ins with PTTASC coach to discuss plan
  • Quarterly updates to document work plan progress/barriers

Comments from some coalitions on the CVAT process

PTTASC experienced positive feedback from the coalitions that used the tool and the coaching process. One coordinator stated: “This was truly a godsend for our coalition. We were in the midst of a difficult chapter and it was difficult to see the path forward. The CVAT clearly defined our strengths, limitations and a plan for improvement. We have implemented all of our action steps and look forward to going through the CVAT process again.” Another coalition expressed how using the CVAT brought their coalition together. “During the CVAT process, we came to a strong consensus from within coalition members on points both in strength and developmental areas.”

Most of the coalitions that have availed themselves to the CVAT process report progress on building the capacity of their coalition. Coalition staff have reported creating a host of new documents including: by-laws, new member on-boarding materials, roles and responsibilities for both members and leaders, committee descriptions and duties, as well as increasing membership and targeted recruitment, improving pre-meeting communications, and using on-line platforms to delegate work to members. PTTASC provided ongoing support for coalition priorities by creating templates that could be adapted for local use such as member and leadership roles and responsibilities, and compiling resource materials, e.g. on-boarding materials for coalitions to use.

Through the CBPR process that allowed coalitions to choose their priorities, coalitions owned the improvements they were seeking resulting in some action steps and improvements in all of the coalitions that used the CVAT. PTTASC’s support to help them achieve their priorities made the action steps easier, increasing outcomes and the successes of the coalitions.

Conclusions

The CVAT is clearly a work in progress, having been underway for approximately 16 months (development and initial implementation). We are encouraged by the feedback received to date; there appears to be increased ownership of coalition operation by members which addresses some of the concerns raised by others (e.g., Goldstein et al., 2017; Tufte & Mefalopulos, 2009) regarding levels of member participation. Future tasks will be the documentation of the sustainability of these and other changes in the operation of the coalitions as well as refinements in the language of the CVAT and exploration of its psychometric properties.

Community coalitions are often created to help communities address complex and seemingly intractable problems, problems that often demand a paradigm shift (Seidman, 1983; Watzlawick et al., 1974) regarding how the community conceptualizes the issues. Such a shift often involves gathering data and perspectives from new sources and reinterpreting their meanings which are, indeed, cognitively and logistically challenging tasks. Coalition success largely depends on the synergy that arises from of its membership, an energy that demands that members understand and embrace the many responsibilities that effective coalition participation demands.

The CVAT has the potential to be a tool that can assist coalitions in understanding the many structural and interpersonal dynamics critical to successful coalition functioning. When wielded by technical assistance consultants who can use it to empower community organizations, we believe it is a technology that can help move these groups toward making make meaningful change in their communities.

For questions on obtaining and using the CVAT, please contact John Daviau at John@cappct.org