Background

Communities continue to deal with persistent health problems despite the enormous investment in health research, service delivery, and program development to address those health concerns. Sometimes, these health problems represent a translational research gap, where there is sufficient medical evidence to address the community’s health needs, but that evidence has not been adequately disseminated or sufficiently implemented within the community (e.g., the translation of cardiovascular disease guidelines or the interaction between depression and diabetes). In response to these translational gaps, community and academic members of our team developed the Boot Camp Translation (BCT) process. Rooted in the principles of community-based participatory research, BCT is an evidence-based process that engages local community members in partnership with academic researchers in translating complex evidence-based guidelines and medical jargon into locally-relevant and actionable health messages and dissemination strategies (Westfall et al., 2013, 2016). However, for many health conditions there is a paucity of evidence-based medical guidelines (e.g., chronic pain management or accessing mental healthcare). There is either no evidence to translate or, most of the time, there is some available medical evidence but not enough for complete translation, dissemination, and implementation. For instance, treating depression is evidence-based, but there is no evidence about how to help an individual access behavioral health services in an underserved rural or urban community.

Appreciative Inquiry (AI) is a form of action research that seeks to identify occasional successful events (also called “positive deviance”), describe the underlying elements and conditions that fostered success, and leverage or replicate those elements and conditions to improve everyday outcomes (Bleich & Hessler, 2016; Hammond, 2013). Positive deviance relies on the premise that there are examples of individual behavior that confer an advantage to that person compared to others (Marsh et al., 2004; Pascale et al., 2010; Tuhus-Dubrow, 2009). While AI has been used in the social sciences for several decades, it has only recently begun to appear in the clinical medical sciences. AI has been used in healthcare to promote healthy behaviors, recognize strengths in a rural community, and improve mental health in elders (McCarthy, 2017; Watkins et al., 2016; Whittaker et al., 2015). Appreciative Inquiry is an attractive approach towards describing positive deviance, encompassing a formal, replicable method to identify, define, and accentuate the elements of local solutions.

The High Plains Research Network (HPRN) Community Advisory Council (C.A.C.) and the Colorado Research Network Patient Partners Research Council (PPRC) have worked collaboratively on community-based participatory research efforts for 15 years. The teams of community members include farmers, ranchers, schoolteachers, students, retirees, laborers, and small business owners. These groups have developed working agreements (Westfall et al., 2013), IRB protocols (Westfall et al., 2017), patient and practice initiatives (Allison et al., 2014), presented at national and international research conferences together, and co-authored manuscripts and book chapters (Norman et al., 2013; Westfall et al., 2020; Zittleman et al., 2020). The groups have spent hours reflecting on their own personal, local, and professional agendas and developed a robust process for assuring equity in power, voice, and resource allocation (Kjellström & Mitchell, 2019).

The advisory councils have used informal AI techniques for several years addressing asthma management, colon cancer prevention, and diabetes (Bender et al., 2011; Norman et al., 2013; Westfall et al., 2016). The C.A.C. recently utilized AI in the development of an awareness campaign around patient-centered medical home efforts in eastern Colorado. The assets-focus of the AI method resonated with HPRN C.A.C. members, who shared their stories of successful interactions or experiences with primary care clinicians and other providers. C.A.C. members solicited additional stories from their friends and colleagues. Through the AI process, the group identified the unique features and components of these successful interactions, which focused messages about patient-centered medical home around the concept of relationship, particularly between the patient and provider, and shaped ideas for disseminating messages at both the community and practice levels (Allison et al., 2014).

There are a number of methods for generating medical evidence, ranging from the classic basic-science-discovery-translated-into-clinical-practice pathway (Califf et al., 2016; Zerhouni, 2003) to the pragmatic practice-based creation of “real world evidence” (Khosla et al., 2018). The key innovation of AI in the healthcare setting is the collection of stories describing people’s best experiences: instances when something “really worked.” The stories of apparently random successes are methodically analyzed and organized into new, locally-generated evidence. However, as with most evidence, even AI-generated data needs translation into practical messages that resonate with community members and can be more widely implemented. Applying this new evidence to the BCT process allows patients and community members to rapidly and efficiently translate the AI successes into locally-relevant programs and sustainable care. By self-defining the local problems and identifying the locally-relevant individual successes, the combined Appreciative Inquiry and Boot Camp Translation (AI/BCT) process can create solutions for the entire community, and lead to sustainable transformation in how communities address their own health problems. If all healthcare is local (Klein et al., 2017), this process can help communities develop local healthcare solutions.

This combination of Appreciative Inquiry and Boot Camp Translation (AI/BCT) provides a formal method for addressing the current gaps in patient-centered outcomes research. Specifically, AI/BCT provides a method for communities to reliably engage their own members, and appropriate healthcare and research professionals, in a long-term collaboration that spans the entire research process from the prioritization of health issues, to the generation of local evidence, to the translation of evidence into locally-relevant messaging and materials. AI/BCT engages patients and community members early and often to: identify local health problems; share community, patient, provider, and academic expertise; and develop and implement patient-centered research and interventions. AI/BCT aims to transform health by making the random the usual.

The purpose of this manuscript is to describe the process for conducting Appreciative Inquiry to identify successful strategies for a priority health topic and apply those findings to the BCT process. This process was developed over five years of informal AI activities, then tested in five formal AI/BCTs that addressed community-identified topics — access to mental health care and support, chronic pain management, sleep apnea, and practice transformation — in underserved rural and urban communities in Colorado. This research was funded by the Patient Centered Outcomes Research Institute and approved by the Colorado Multiple Institutional Review Board.

Methods

Selecting a topic for appreciative inquiry boot camp translation

Our AI/BCT process begins with communities identifying a health topic of interest to them. We use a community engagement approach, the Delphi Method, to identify the health care issues of importance and priority within each community and combine it with a review of the academic evidence on the community-identified topics to determine need, feasibility, and current evidence base. The Delphi Method is a widely used and accepted technique well-suited for consensus-building. We use a modified Delphi Method (Crabtree & Miller, 1999) with the goal of using an iterative process with community members to narrow topics to a practical priority list for which there is general consensus (not unanimous agreement) (Addison, 1999).

Participant selection is often a concern when beginning a Delphi Method assessment. We use multiple methods to identify a broad range of community members to complete individual priority-setting in each community. For communities or organizations with a current community advisory council or board, an initial list of health topics can typically be obtained quickly at a regularly-scheduled council meeting. For those without an advisory board, a broad range of community members can be invited to a “town hall” style meeting using convenience or snowball sampling. For the latter, this group can be as small as 10–15 community members or as large as 100 or more.

For larger groups, table seating will provide better conversation than classroom seating. Organizers should allow two to two-and-a-half hours to brainstorm health issues and serve a meal or snack, if possible. The final product should be a list of 10–20 health issues important to the community.

From the initial list of health concerns, the academic partner organizes and refines the list. Some issues will be beyond the scope of the current partnership, and these may be removed from the list or included with a caveat that, while important, may not be able to be addressed (e.g., a new hospital, providing free medication for people with chronic diseases, etc.). The academic partner also reviews medical evidence available for the health issues. Topics for which evidence-based guidelines exist are not designed for AI/BCT and can be removed from the list. The academic partner distributes the initial unranked list to participants via email or regular mail and instructs participants to choose the top five health issues they personally believe need to be addressed in their community. The academic partner compiles the results, and the ten health topics receiving the most votes are distributed again. At this stage, the health topics are listed in random order, not by rank. This provides the greatest opportunity for community members to make fresh choices. Participants are again asked to identify their top five topics, results are compiled, and the list is distributed — this time in rank order. If possible, groups may meet again in 2–4 weeks for one hour to review the priority list and offer input on which topic to consider for AI.

After these rounds of prioritization, feasibility becomes an important consideration. Are there stories of successful management of the priority topic? For instance, one community identified the rise of sexually transmitted disease in the young adult population as a priority topic. However, the community felt it would be difficult to obtain success stories for this topic. In our study, we were surprised by the community prioritization of mental and behavioral health. While the academic staff had concerns about finding success stories about a stigmatized health topic, our communities gravitated towards this topic because of its high prevalence and community impact. Ultimately, finding stories of successfully accessing mental health care or support in their communities was not difficult.

Communities do not have to agree on a singular priority topic. For long-term community partnership building, we establish an agreement to consider the top few priority issues in the coming years which keeps people engaged and provides opportunity to address one issue at a time. Other considerations for choosing the final topic for AI/BCT include local and national funding opportunities and the skills and interests of the associated research team. To provide guidance, Table 1 offers a list of necessary elements for AI/BCT topic selection. Partnerships should choose one of the top three or four priority topics. Celebrate the choice and get the community group or advisory board excited about continuing the work together.

Institutional review board (IRB) and human subjects considerations

The study protocol was approved by the Colorado Multiple Institutional Review Board. Verbal informed consent was obtained from each AI interview participant.

AI/BCT presents the opportunity to engage community member partners as participants in the analysis of the AI data. Each AI/BCT brought together a community-academic partnership, including 10–15 community members, many of whom were involved in the topic selection process. Community members were partners in the research and received COMIRB-approved human subjects’ research training. As described elsewhere, multiple methods can help assure community partners have the appropriate knowledge, understanding, and certification in human subject protection and privacy (Westfall et al., 2017).

Recruiting people with stories of success

We sought to find people in the community who had stories of success around the chosen topic. AI is grounded in the concept of success. That is, what worked for the individual or group to address the identified problem? We sought out community members and patients who had stories that, while they might include initial struggles and challenges, ultimately contained elements of success and positive outcomes. The study population for the AI research process is ideally recruited using snowball sampling through community partners, collaborators, participating clinics, staff, and providers. We used our community research liaisons and community-based organizational partner staff to assist in AI participant recruitment. Health extension agents (in Colorado, they are called regional health connectors), community health workers, primary care extension agents, and community advisory boards can also help recruit people with stories of success. Flyers describing the health topic and AI process placed at community health centers, churches, healthcare practices, and other busy locations were effective ways to reach potential participants.

Upon initial identification of interest and willingness to participate, the research staff conducted a brief 5–10 minute phone screening process with the potential participants to ascertain whether their story would include elements of success relevant to the health topic and provide valid data for the AI process. These elements, or markers of success, are explicitly defined by the community-academic project partnership. For example, success in addressing chronic pain was defined as: “being able to do most of what one wants or needs to do on most days of the week.” For the mental health topic, after initial screening by staff, the research team reviewed de-identified story briefs and chose a final pool of participants for AI interviews. Data collection should continue until emergence of new components of success slows, rather than adhering to a specific number. However, our experience was that approximately 15–30 stories were necessary to represent the salient issues and be certain no big ideas were missed.

Conducting appreciative inquiry interviews

Study participants should be people who have successfully managed the topic chosen by the community. Group or individual interviews may be used to collect stories of success, depending on the nature of the selected topic and scheduling needs. During the first two AI/BCTs, we used a combination of group and individual interviews. We found that individual interviews delivered more robust, individually-focused stories of success and chose to use only individual interviews for the remaining three AI/BCTs.

To assure compliance with all privacy and confidentiality regulations, trained and experienced interviewers should conduct the interviews. Prior to conducting any interviews, the team needs to develop a semi-structured interview guide to help ensure consistency across interviews conducted by various team members. The interview guide provides one or two key questions to solicit the positive aspects of the story (Hammond, 2013). Table 2 provides an example interview guide. For our work on successful strategies for accessing mental health care, we asked the following lead questions: What was the care or support you received for a mental health issue?; What led up to it?; What factors made access possible or likely?; What factors made access difficult? Probing questions helped discover demographics and characteristics of the people involved with the story such as age, gender, insurance status, family relationships, employment situation, role in the community, and financial situation. As participants may revert to barriers or negative story elements, the AI interviewer guides them back to what worked and elements of success.

For groups, participants are assigned a number and directed to refer to that number prior to speaking to create anonymous data. Participants respond in turn by describing their own individual story. Clarification may be requested by the interviewer or other participants as the individual shares his/her story. A group discussion of any questions or insights occurs toward the end of the session. For individual interviews, the same questions are asked, and probing follow-up questions keep the story aimed at the AI theme of “what worked for you.”

Group interviews lasted from 30 to 120 minutes and individual interviews from 30 to 60 minutes. We provided participants with an incentive (e.g., $50 gift card to a local grocery store or other retailer) for their participation. All collected data was audio recorded. Interviewers took notes on the content and their impressions of the discussion. The audio recordings were transcribed, cleaned, and placed into a qualitative software program (e.g., ATLAS ti, version 7; Scientific Software Development, GmbH, Berlin, Germany). Each participant was asked to complete a demographic form upon interview completion (e.g., gender, age range, and race/ethnicity).

Analyzing the appreciative inquiry data

While the AI analysis process begins with traditional qualitative methods using a grounded theory approach (Addison, 1999; Crabtree & Miller, 1999), it includes multiple steps to engage the broader patient and community partners. A team of three qualitative researchers first reviewed the transcripts to develop a broad, initial coding scheme. This team completed an immersion-crystallization phase in which they reviewed the data to identify themes for organization (Borkan, 1999), develop a specific coding structure, code text segments according to this structure (Crabtree & Miller, 1999; Miles, M. B., Huberman, A. M., & Saldaña, J., 2014), identify patterns and relationships across codes and themes, connect themes and patterns to existing knowledge (Miller & Crabtree, 1999), and then corroborate/legitimate to seek out additional data to confirm or refute insights from the initial analysis.

More specifically, the core qualitative team members first read the transcripts separately to get a general sense of the responses and key issues emerging from the data using a grounded theory approach. They discussed a series of codes to represent the themes. They coded several transcripts together to achieve conceptual overlap in rating before dividing up the remaining interviews to individually code them all according to the agreed-upon structure. We found this stage of analysis greatly benefitted from having at least one of the analysts involved with the collection of AI data. After initial coding, the analysis team expanded to a larger research team for more thorough thematic analysis. This larger group may include team members who conducted interviews, other researchers, community members, patients, or providers. Special caution is taken to preserve the confidentiality of the interview participants during this process. The additional members discuss the coding elements and early results. Emergent themes are shared iteratively in summary format over a series of meetings with the larger research team and community councils to ensure accurate interpretation of interview data.

Next, the core analysis team uses a formal editing approach to determine themes within and across stories. The key questions are: What are the higher-level ideas that we can take from these stories? What do the stories say collectively? Thematic summaries are developed and shared with the overall research team. Finally, in order to identify patterns to the stories, i.e., similarities in the strategies, outcomes, or participant characteristics, the core team may find a template approach to organizing the data useful, reviewing transcripts to capture all excerpts relevant to the previously identified themes.

At this point, additional investigators and community members (including several who have not been part of the initial reviews) join the research team to create a matrix display of themes (Miles, M. B., Huberman, A. M., & Saldaña, J., 2014), a table representing all the stories using the agreed upon codes as an organizing framework. We call this the “big data table” because it has a significant amount of data and level of detail and is presented in a large table format, often on oversized 11"x17" paper (see Figure 1). The table is structured to identify key core issues for all stories. Each story occupies a row in the table, each column represents an identified theme, and table cells list core issues identified in each story, according to information available in transcripts. Using the “big data table,” the research team — which includes patients and community members — spends several sessions reviewing the data and narrowing the content to identify consistent patterns and develop a reduced-size table containing the most important components of each story. Stories are then grouped together by common central elements. For example, in our rural AI/BCT on access to mental health care, several stories included elements of young children’s behavioral issues noticed by schoolteachers, so these stories were grouped together according to commonalities in age of the person seeking care and particular symptoms or behavior. Other success stories included a friend or community member who noticed behavior changes and encouraged the person to seek help.

Figure 1.A Portion of the “Big Table” – Stories, themes, and patterns

The entire AI/BCT research team does not need to be skilled in AI data analysis. However, selecting analysts that possess an advanced skill set is important as the analysis of AI data and its application to BCT involve subtle nuances that differ slightly from more common qualitative data analysis, such as coding, identifying thematic reviews, and organizing results require an analyst that remains in the “what worked” frame of mind. Further, identifying themes and organizing data for BCT require an analytic lens that comes from a solid understanding of both AI and BCT. Table 3 describes our recommendations for the skills and characteristics the AI data analyst might possess.

Reporting the results of appreciative inquiry

Reporting the results from the AI is an important next step. These results provide the evidence base for the expert presentation provided during the initial phase of the BCT process; therefore, we carefully considered how to report the results. The final results are reported to the entire study team, which includes all the core and ad hoc patient and community partners, for discussion and reflection. Presentations and reports include the number and demographics of AI participants and the number of stories, since multiple participants may describe the same specific story, or one participant may offer more than one story. A description of the codes used during AI data analysis is helpful to the BCT partners. Also presented is the final table of distilled, de-identified stories with key elements of success, a list of the key elements of success with a definition of each and examples pulled from the stories, and the overall themes from the AI data. De-identified quotes can be used to illustrate the elements and themes. In the rural mental health AI, a theme emerged of an outside person (not family) that identified behavior changes and engaged the person in conversation and recommended getting help or guided them to care. A second theme related to getting care early before a mental health problem had deteriorated into serious disability or potential for self-harm. Findings are finalized by the advisory councils and academic partners who jointly select the crucial elements from the appreciative inquiry stage for translation into practical messages, calls to action, and materials that resonate with the community in the Boot Camp Translation phase.

Transitioning from the appreciative inquiry process into Boot Camp Translation

A smooth transition from the AI into the BCT process is critical. The AI findings are incorporated into the BCT process as the new, locally-generated evidence base for the selected topic. A description of BCT is provided elsewhere (Norman et al., 2013; Westfall et al., 2016). Briefly, Boot Camp Translation begins with a full-day kick-off meeting comprised of a 2-to-3 hour expert presentation using the national evidence-based guidelines and recommendations for the clinical topic of interest. This is followed by several hours of questions, answers, and brainstorming the important elements of the expert presentation. Over a 4–6-month period, using a mixture of short phone calls and longer in-person meetings, the BCT group translates these key elements into local language, constructs, terms, and communication methods to incorporate into community and practice messages and materials. BCT translates complex evidence into locally-meaningful programs and messages. It is not a rhetorical process that simply takes evidence-based guidelines and published recommendations and changes a few medical words. BCT alters the conceptual framework that community members and patients may hold for the specific medical topic. BCT groups don’t know what they are going to end up with when they start the process. Because BCT uses local experts, once educated about a specific health topic, they have the capacity and local knowledge to frame the condition in the community milieu. In the same way, AI-generated evidence in its raw form is unsuitable for dissemination and rapid uptake within a community. Our AI finding that a critical step in accessing mental healthcare was an outside individual reaching out required transformation by the BCT process into a community-designed program to encourage and give community members the tools to more routinely perform exactly this kind of check-in.

For AI/BCT, we slightly altered this half-day expert presentation to include two parts. First, an expert in the health topic provides a presentation on what is known about the given topic, specifically noting why there are a lack of existing guidelines or other barriers around implementing guidelines into everyday practice. This grounds the BCT participants in the topic and explains why an AI approach to developing local evidence was needed. Second, an expert in the AI data presents the AI results, i.e., the new local evidence, to describe the local strategies for success. This presenter is often one of the lead qualitative analysts who was immersed in the data analysis.

In addition to the usual BCT process of participants asking detailed questions to unpack the science of the evidence, AI/BCT participants might pose questions that suggest new avenues of inquiry into the AI data. For example, our chronic pain management BCT participants sought more information about why opioids were not more important to successful management of our interviewees’ chronic pain. This community-initiated inquiry of the AI data yielded important additional insights.

How to implement the BCT products

The BCT process identifies how to make what was apparently the random the frequent. As with any BCT, celebrating the resulting set of messages and intervention materials is an important step. Ideally, projects that use the AI/BCT method should include funds to produce, implement, and evaluate the resulting messages and interventions. However, describing the implementation and evaluation of the interventions is beyond the scope of this manuscript.

Discussion

When health issues lack evidence-based guidelines, or existing guidelines are not relevant or applicable to a particular community, individual community members must deal with the health issue the best they can. Sometimes, they get it right and have a successful outcome. Our community partners tell us they often lack the ability to identify and replicate isolated examples of success into approaches that might work for their whole community. Because the AI/BCT approach focuses on recreating the components real people have utilized to solve their problems (positive deviance), rather than imposing a top-down generated solution, we believe the likelihood of success is increased. In the specific example of rural mental health access, several components of success were identified that appeared to work together to produce what the participants perceived as a successful outcome. The ideas of early intervention, and the “other person” engaging and asking questions as components of individual success were key elements of the “evidence” common to successful stories related to mental health. With the AI/BCT process, community members in the BCT were able to share their knowledge of how to reach farmers, ranchers, and community members with messages and materials to educate and motivate them to act.

Analyzing AI data for research is different than standard AI for strategic planning. Typical AI for strategic planning and program development can move quickly through the success stories related to the business, program, or activity. However, in identifying the successful components related to health issues, the stories are often more complex, requiring much more in-depth analysis. For example, in both AI/BCTs on accessing mental health care, the stories frequently included the actions of a third party, often not a family member, but some other person in the community. This concept of the “other person” might have been missed by a simple review and thematic analysis of the stories. It was through the in-depth qualitative analysis and review by both academic and community members that this concept of the “other person” was identified. The qualitative team brought this idea to the whole research team and partnering community advisory councils and heard immediate confirmation of the validity of this finding. In the BCT phase of the rural mental health project, the individual success themes were translated into a program called Changing Our Mental and Emotional Trajectory (COMET) (Figure 2). COMET includes public education and a five-question guide when engaging with a person who seems to have some behavior changes or is suffering (see Table 4). COMET training has been conducted for numerous organizations and several hundred community members. Initial evaluations from those trained in COMET are positive for increased knowledge and intent. COMET is now being implemented throughout rural Colorado as well as several other states.

Figure 2.COMET Conceptual Diagram Created in BCT

AI/BCT can be applied to a host of health topics. Clinical topics with inadequate or no evidence-based guidelines or recommendations are excellent candidates for AI/BCT. Topics such as chronic pain, sleep apnea, and depression in patients with diabetes have minimal evidence-based identification, treatment, and management guidelines. Some have evidence limited to just a portion of the overall condition that have not been coalesced into a formal guideline. For example, there are numerous guidelines for use of opioid pain medication in patients with chronic pain. However, there are few or no guidelines that provide evidence-based approaches to maximizing treatment for chronic pain with other treatment modalities (physical therapy, counseling, non-opioid medication, exercise, etc.). The social determinants of health are commonly identified by community members as barriers to health and healthcare. Very little national evidence exists for improving health through interventions aimed at the social determinants of health. Remarkably, community members have individual stories of success. Promising uses of AI/BCT are to address safe housing, food security, and education. Systemic change is also amenable to AI/BCT. Few evidence-based guidelines and recommendations exist for addressing important local system change. Topics such as practice transformation, network development, an individual’s medical home base and neighborhood are rich with individual success stories and others in search of making those random successes the usual practice. The AI/BCT method we have described provides communities with a standard process for identifying and addressing their own gaps in evidence, transforming the local solutions, creating a sustainable model for health improvement. The process is an important addition to the standard translational pathway and supports bidirectional movement of health concerns, clinical topics, and research data (see Table 5).

There are many methods for creating new evidence. Our research team works with basic and clinical scientists to discover new treatments or new practice processes. Community and practice-based research include the full spectrum of translational research (Westfall et al., 2007; Westfall & Mensah, 2018). However, when there is just not adequate evidence, we have found that the AI/BCT process can fill a gap and provide local communities with an assets-based approach to improving health. The AI/BCT process identifies individual examples of positive deviance, carefully distills the crucial elements of those narratives using local community members as experts, and translates the elements derived from individual success stories into local implementation of messages, materials, and programs. AI produces local data from individual stories. These success stories often have common elements, but these common elements need to be brought together into a broader intervention with more general community messaging, materials, and programs. Engaging patients and community members in the BCT process assures ideas and programs are locally relevant. As an added bonus, focusing on what works is more enjoyable for researchers and participants.

Conclusion

This study tested the feasibility and promise of AI/BCT to address gaps in evidence to address locally relevant community and patient health concerns and issues. We were able to distill key components of the process and provide a description of how to successfully conduct an AI/BCT. Appreciative Inquiry coupled with Boot Camp Translation engaged community members, patients, organizations, and academic partners throughout all aspects of the process. We believe the method is ready for use by other academic and community/patient partners on other health topics, and we encourage others to seek funding to support application of the method.

Declaration of Conflicting Interests

The Author(s) declare(s) that there is no conflict of interest.

Funding

This work was supported by the Patient Centered Outcomes Research Institute [grant/contract number ME-1303-5843].