Introduction
Translating health-related scientific discoveries into meaningful clinical and public health impact is a slow and uncertain process, taking an average of 17 years (Morris et al., 2011) if it occurs at all. Engaging the stakeholders affected by research results–including patients, caregivers, community members, and clinicians—is a promising strategy to improve the relevance and impact of clinical and translational research (CTR) (Selker & Wilkins, 2017) and to support dissemination and implementation of results (Brett et al., 2014; The Value of Engagement, 2023). This “broadly engaged team science” approach (Selker & Wilkins, 2017) opens space for community members and other stakeholders [hereafter ‘stakeholders’ (SHs)] to contribute insights from lived experience, contextual knowledge, skills, and awareness of unmet needs that can shape research questions, interventions, and the selection of study outcomes that matter most to patients and communities (The Value of Engagement, 2023).
Much of what is known about the potential benefits, challenges, and strategies for effective broadly engaged team science is based on well-funded, longer-term research projects. For example, the Patient Centered Outcomes Research Institute (PCORI), which requires participation of community members and other stakeholders throughout projects (Engagement Resources, 2021; Forsythe et al., 2016; Frank et al., 2020), funds multi-year projects with a median award amount of ~$300,000 between 2012-2023 (Explore Our Portfolio, n.d.). Even pilot projects funded by PCORI have lasted two to six years.
For longer-term projects funded by PCORI and others, existing literature posits that engaging stakeholders enhances research design, study materials, recruitment, research conduct, and results interpretation; improves research quality and cultural relevance; and facilitates more meaningful dissemination into communities (Brett et al., 2014; Brodt et al., 2015; Cukor et al., 2016; Forsythe et al., 2016; Israel et al., 1998; Morgan et al., 2022; Pinsoneault et al., 2019). In these larger studies, previously reported challenges to achieving these benefits, and strategies to overcome them, relate to team building and cohesion, limited time and resources, training and skill-building, and broader cultural elements.
Yet, many larger health-related studies evolve from smaller proof-of-concept or short-term pilot studies, and engaging stakeholders in these initial studies can help shape the direction of subsequent larger studies and maximize the potential benefits of engagement and the study. Building on prior evidence from longer-term research, this article examines CSE in a different context: early-stage, modestly funded pilot projects across the translational spectrum. By describing the strategies, experiences, and perspectives of investigators and stakeholders involved in these initial projects, we address the research question: What are effective practices and strategies for engaging stakeholders and community members in early-stage, modestly funded research that shapes larger studies? Results will inform investigators, their affiliated organizations, and funders about strategies for embedding meaningful stakeholder engagement from the start of translational research.
Known Challenges and Strategies of Implementing CSE in Longer-term Research
Existing literature, focusing on longer-term studies, identifies four categories of challenges to and facilitative strategies for implementing effective engagement: i) team building and cohesion; ii) time and resources; iii) training and skill-building; and iv) broader cultural elements.
First, team building and cohesion in the context of implementing CSE bring challenges that begin with identifying the pertinent stakeholders and achieving representativeness among stakeholders (Ashcroft et al., 2016; Boote et al., 2011; Brett et al., 2014; Concannon et al., 2019; Cukor et al., 2016; Forsythe et al., 2016). Additional challenges range from navigating conflict (Ashcroft et al., 2016; Boote et al., 2011) and mistrust between investigators and stakeholders (Forsythe et al., 2016; Israel et al., 1998) to setting clear expectations as well as sustaining participation over time (Brett et al., 2014; Concannon et al., 2019; Cukor et al., 2016; Forsythe et al., 2016).
Known strategies to build broadly engaged teams and increase cohesion begin with thoughtfully identifying stakeholders early in study development and demonstrating to them the value of engagement (Brodt et al., 2015; Cukor et al., 2016; Forsythe et al., 2016; Israel et al., 1998; Pinsoneault et al., 2019). As the research progresses, investigators can create a written engagement plan (Concannon et al., 2019), clearly define stakeholder roles and expectations (Brodt et al., 2015; Cukor et al., 2016; Forsythe et al., 2016), co-create group norms and goals (Israel et al., 1998), conduct regular meetings (Pinsoneault et al., 2019), adapt communications to meet stakeholder needs (Forsythe et al., 2016), promote shared leadership (Forsythe et al., 2016; Israel et al., 1998; Pinsoneault et al., 2019), and demonstrate an appreciation for stakeholders’ input (Israel et al., 1998).
Second, implementing CSE may necessitate additional time, resources, and complexity (Boote et al., 2011; Brett et al., 2014; Concannon et al., 2019; Forsythe et al., 2016; Israel et al., 1998; LeClair et al., 2020; Nease et al., 2018), including funding to support engagement and compensate stakeholders for their time (Ashcroft et al., 2016; Brett et al., 2014; Concannon et al., 2019; LeClair et al., 2020). Additional potential challenges include short timelines for integrating stakeholder perspectives (Ashcroft et al., 2016), such as for grant submissions (Cukor et al., 2016), and navigating conflicts over how funds should be spent (Israel et al., 1998). In contrast, funding to support community activities and stakeholder remuneration is a powerful strategy to successfully implement CSE (Boote et al., 2011; Forsythe et al., 2016; Israel et al., 1998; Nease et al., 2018; Pinsoneault et al., 2019).
Third, investigators and stakeholders may lack relevant training, expertise, and skills. Many investigators lack an understanding of or mentorship in CSE, its best practices, and cultural sensitivity (Boote et al., 2011; Forsythe et al., 2016; Israel et al., 1998; LeClair et al., 2020; Nease et al., 2018). Likewise, stakeholders often lack training in research methods and a full understanding of the specific research topic (Boote et al., 2011; Brett et al., 2014; Cukor et al., 2016; Forsythe et al., 2016; Israel et al., 1998). Building investigator and stakeholder capacity in these areas supports engagement (Boote et al., 2011; Forsythe et al., 2016; Israel et al., 1998; Nease et al., 2018); it is important for investigators to clearly communicate the research topic and scientific methods to stakeholders (Boote et al., 2011).
Finally, broader cultural contexts can support or impede CSE implementation. Challenges can include cultural and social dynamics of the community (Boote et al., 2011; Israel et al., 1998), power imbalances between investigators and stakeholders (Brett et al., 2014), and misalignment between CSE and traditional academic promotion criteria that do not account for the time involved in CSE implementation and rewards independent research (Israel et al., 1998; Nease et al., 2018). Contexts supportive of implementing CSE include local expertise in engagement methods, existing connections between investigators and stakeholders, and a research culture that supports engagement (Nease et al., 2018). Standard research practices supportive of CSE, or lack thereof, can affect its success, e.g., availability of an “IRB protocol that could be used for community partnerships” and standard dissemination processes that include community partners (Nease et al., 2018).
Tufts CTSI Pilot Studies Program and Participatory Requirement
Given the posited benefits of engagement to research quality and relevance among longer-term studies, Tufts Clinical and Translational Science Institute (CTSI) leveraged its role as a funder and research support organization to catalyze engagement early in research conceptualization, i.e., proof-of-concept studies or preliminary data collection to support funding a larger follow-on study. Tufts CTSI believed engagement at those early stages was important to define the direction of larger subsequent studies by aligning with the needs and priorities of stakeholders to increase studies’ relevance and potential impact.
In contrast to larger, longer-term studies, since 2009 Tufts CTSI has funded early-stage, modestly funded pilot projects, each with funds ranging from $30,000 to $60,000 for a one-year duration. The goal of the Pilot Studies Program was to provide seed funding for proof-of-concept studies or initial assessment of innovative ideas across the translational spectrum, from early testing for safety to studies with patients and public health initiatives. Concurrently, Tufts CTSI’s Community and Stakeholder Engagement Program developed frameworks, guidelines and approaches to support community engaged research and broadly engaged team science (Concannon, 2015; Concannon et al., 2012; Concannon, Fuster, et al., 2014; Concannon, Guise, et al., 2014; Hacker et al., 2012; Rubin et al., 2014; Rushforth & Selker, 2022; Selker & Wilkins, 2017).
Recognizing the synergies of these two initiatives, Tufts CTSI established a requirement that its funded pilot studies include engagement of community members and other stakeholders. To assist investigators in meeting this requirement, the hub provided written guidance and consultations about implementing broadly engaged team science. Importantly, this framework recognizes that engagement cannot be formulated as a singular approach for every research project. Instead, a fit-for-purpose approach is essential to incorporate a variety of stakeholders in different roles as appropriate for the translational stage of the research (Rushforth & Selker, 2022). The program expected awardees to engage stakeholders effectively in their research, including evolving engagement strategies in response to stakeholder feedback and emerging needs.
This article describes the experiences with and strategies for implementing participatory research among Tufts CTSI Pilot Studies Program awardees. To our knowledge, this article is the first to report insights from both investigators and stakeholders about engagement in initial, modestly funded health-related projects across the translational spectrum that are designed to inform follow-on research or practice.
Methods
Overview
An evaluation team independent of the Pilot Studies Program led a mixed methods study about the new CSE requirement, with the dual goal of informing program improvements and contributing to existing literature. The study elicited perspectives and experiences across three annual Pilot Studies cohorts (2019–2021). The current report focuses on strategies and recommendations of awarded teams (Principal Investigators (PIs) and Stakeholders (SHs)) regarding implementing CSE during the award period. The evaluation study was reviewed and approved by the Tufts Health Sciences Institutional Review Board.
Participants and Recruitment
Study participant recruitment occurred when the funding period for each pilot project ended. A Tufts CTSI evaluator invited each PI to participate in an interview about their experience implementing CSE. PIs were also asked to identify one SH who was involved in project activities to be invited for an interview. Both PIs and SHs received an initial email invitation, followed by two reminder emails, a phone call, and a final reminder email, as needed. For PIs, evaluators also contacted their administrative assistants.
Eleven pilot awardee teams received extensions of up to one year to complete their projects, due to the COVID-19 pandemic (63%) or other unforeseeable delays sometimes in combination with COVID-19 delays (36%). In these cases, recruitment was delayed until the end of the extension period to retain consistency in conducting interviews at the end of each project’s funding period. Recruitment occurred from August 2020 to August 2022.
Data collection
Two semi-structured interview guides, one for PIs and one for SHs, were developed to elicit information about implementing CSE. Both guides addressed a range of topics, including facilitating strategies and challenges; recommendations to investigators, research support organizations, and funders for enhancing CSE implementation; and perceived meaningfulness of CSE. Additionally, the PI guide focused on how stakeholders were involved (in planned and unplanned ways), and the SH guide explored recommendations for how other stakeholders can make participation meaningful.
Each interview was conducted by an evaluation team member who had previous training in qualitative methods. To optimize data quality, evaluators participated in study-specific mock interviews before conducting interviews and in team reflection sessions during the data collection period. All interviews were conducted via videoconference and, with participant permission, recorded and transcribed for accurate analysis. PI interviews averaged approximately 26.7 minutes (based on 13 recorded interviews; one interview was documented through notes rather than recording, and interview length is unknown), and SH interviews averaged approximately 26.2 minutes. Interviewees did not receive compensation or other incentive to participate.
Data analysis
The evaluation team utilized an inductive process to analyze interview data. Two analysts independently coded an initial batch of transcripts and developed independent draft codebooks, met to compare their coding, and developed a working codebook with codes and definitions (Lofland & Lofland, 2009). Throughout the coding process, analysts independently coded interviews, meeting regularly to reconcile their coding and revise the codebook as needed. As new codes were added and existing codes were modified, previously coded interviews were reviewed to ensure coding matched the most recent codebook.
Next, the team conducted thematic content analysis by exploring connections between codes and using memos to elucidate emerging themes (Green & Thorogood, 2009). Particular attention was paid to the most frequently used codes. The team then organized coded segments by theme and compared themes across PI and SH groups to identify commonalities and differences. The team organized results into methodological domains relevant to participatory research practice, including: facilitating strategies; barriers; and recommendations to funders, research support organizations, and other investigators.
To describe how CSE was implemented across pilot projects, three categories of SHs were defined: those external to the study team, those who were study participants, and those who were study team members (Table 1). This categorization was intended to elucidate the types of engagement only. The analysis assumed all categories were of equal value and approached data from PIs and SHs across all categories in the same manner.
Results
Overall, engagement in short-term, initial pilot projects was feasible and operationalized as a set of interconnected practices and strategies. Across a range of project types, project teams enacted engagement, including studies at both early and later translational stages. The findings highlight that engagement is shaped not only by investigator and stakeholder practices, but also by the cultural and infrastructural conditions that support them.
Participant Characteristics
A total of 19 PIs and SHs participated in interviews about engagement practices and strategies across 14 pilot projects (Table 2). Fourteen out of 24 PIs (58%) participated, and 12 of those spoke to their levels of experience. Most were experienced investigators, but more than half reported no or little previous experience with implementing CSE. Five SHs from five distinct projects participated in an interview.
Project Stakeholders and Translational Stages
The 14 projects engaged SHs with a range of backgrounds and roles, and represented research across the translational spectrum (Table 3). By the end of each project’s one-year funding period, most (n=12, 86%) had achieved participation of stakeholders in some capacity, and PIs for the two remaining projects (14%) planned to do so.
Engaged SHs were patients, patient groups, clinicians (e.g., neurologists, ophthalmologists, occupational therapists), organizational representatives, and community members. Based on our definitions, SHs were included most commonly in roles external to the study team, followed by including them as study team members and then as study participants. This pattern held for projects for which CSE implementation was complete at the time of the interview and those for which CSE was planned during post-funding work on the project.
At the time of the interview, SHs had been involved in various phases of the pilot study, including planning (64%), study conduct (50%), recruitment (50%), interpretation or dissemination of results (29%), and efforts to achieve future funding (14%).
CSE appeared to vary by translational phase (Figure 1). The two projects for which PIs had yet to engage stakeholders at the time of the interview were among the nine representing earlier stages of translation (T1-T1.5), in which stakeholder participation in conducting research can be perceived as more difficult.
These earlier translational stages focus on early research development and feasibility, and CSE that occurred was concentrated primarily in idea generation, study design, and study conduct. In these projects, clinicians and other SHs were most often engaged as members of the research team, contributing to protocol development and early-stage refinement. On the other hand, pilot projects at later translational stages (T2–T4), which emphasize testing, implementation, and dissemination, more often engaged SHs external to the research team, such as organizational representatives and community partners, in recruitment and anticipated dissemination activities.
As many projects were not yet fully completed by the time of the interview, eleven PIs (79%) indicated plans to further engage stakeholders to help interpret or disseminate study results, even though funding had ended. Looking to future research studies, all PIs intended to involve stakeholders in their future work, especially to ensure research would be useful to patients and the broader community and to support recruitment and interpretation of results.
Facilitating Strategies, Barriers, and Corresponding Recommendations
Facilitating strategies
Respondents identified three main facilitating strategies for successful CSE implementation (Table 4). First, initial CSE was catalyzed by drawing on existing networks and seeking stakeholders with a connection to the study topic. PIs employed diverse strategies to garner interest among potential stakeholders, including utilizing investigators’ personal and professional networks, establishing organizations or patient groups, and using flyers and formal phone scripts (7 PIs). As a PI said, “if we were starting from scratch or if I had to do this solo, that would’ve been a lot harder” (PI 6).
Additionally, stakeholders had greater buy-in or commitment when they participated in research that aligned with the mission of their organizations, their personal aspirations, or their professional goals (3 PIs, all SHs). For example, a SH in an advocacy role noted the value of being familiar with the research. “I need to be able to back up what I’m saying in some way…and I need to be able to push people in the right direction” (SH B). Alignment with the mission of a stakeholder’s organization paved the way for straightforward administrative processes, access to organizational resources, and the ability to involve co-workers in the research.
Second, sustained and valuable CSE was supported by building a collaborative team, ensuring clear PI communication, and drawing on relevant stakeholder expertise (4 PIs, all SHs). Proactive efforts to “break down silos” (PI 13) inhibiting stakeholder involvement in academic/medical spaces supported collaboration (2 PIs). Effective communication by PIs, encompassing both verbal exchanges and written materials (e.g., meeting agendas and minutes), facilitated the preparation and active engagement of stakeholders (3 SHs). SHs highlighted the value of structured communication, including materials provided ahead of the meetings that were “well broken down” (SH C) and easy to reference, as well as clear guidance that “lay[s] out what needs to be done” (SH D). One SH underscored the importance of clear communication, attributing the lack thereof to a diminished capacity to engage effectively in the study. Additionally, leveraging stakeholders’ expertise and knowledge pertinent to the study, particularly in areas such as study recruitment, fostered sustained and impactful CSE (3 PIs, 3 SHs).
Third, accessing CSE expertise and resources while conducting the study provided scaffolding for success (8 PIs). Sources of this expertise included Tufts CTSI staff and resources posted on the Tufts CTSI website (6 PIs) as well as guidance from study team members with CSE experience, such as informing decisions about remuneration for stakeholders (2 PIs).
Barriers and recommendations to strengthen CSE
Four barriers limited feasible and meaningful engagement, and respondents recommended four approaches to strengthen and expand CSE (Table 5). These recommendations offer solutions to the barriers experienced while implementing CSE and a vision for developing a research infrastructure that better supports participatory research.
The first barrier revolved around conducting effective team meetings, often because implementing CSE necessitates larger groups (4 PIs, 3 SHs; Table 6). Coordinating schedules and finding mutual availability among investigators and stakeholders within a team was more complex with larger groups. Virtual meetings were necessary to help with scheduling challenges but had important limitations for creating comfortable and safe spaces to build and maintain rapport among team members and to support sustained attention.
Among the four recommendations to strengthen CSE, the suggestion that investigators focus more time and energy on building authentic, sustained relationships with stakeholders (6 PIs, all SHs; Table 7) can help mitigate the limitations of virtual meetings. Developing rapport and mutual understanding by listening and getting to know stakeholders before beginning a research project provides a foundation for effective group dynamics and active engagement in a virtual setting. This also supports meaningful stakeholder contributions early in the research process, including “involving those stakeholders from the beginning in terms of research design” (SH A). In turn, collaboration early in a study signaled that the investigator values the stakeholders’ perspectives and expertise, further strengthening buy-in, despite the limitations of virtual settings. Effective PI communication throughout the study, including clear and understandable expectations (2 PIs, 3 SHs), is key for supporting stakeholder engagement and buy-in. As a SH explained, “…the communication just needs to be as clear, and concise, and simple as possible. …There might be too much scientific jargon or too much stuff to have to wade through” (SH B).
Building and sustaining authentic relationships also provides investigators with insight into what makes participating in a research team valuable and meaningful to a stakeholder, which could help to bridge any important gaps in perspectives. Specifically, in comparing SHs and PIs, a disconnect was apparent in their views of what makes engagement meaningful (Table 8). SHs expressed that gaining new experiences and connections while participating in research gave them an alternative perspective that sparked personal growth, created opportunities to learn from study-related interventions that informed how they conducted their own work, and helped address community needs or raise awareness about a topic. As one participant reflected, “In terms of relevance to the [program] work that we do, it’s pretty significant because it could contribute to program development for us in our…prevention activities” (SH A). In contrast, a majority of PIs believed that stakeholders derived meaning from science itself, i.e., contributing to the advancement of scientific knowledge about health issues (9 PIs). As one PI put it, “[Stakeholders are] all motivated to help the next generations, and so I think there’s a lot of appreciation…for what we’re trying to do, which is develop better treatments” (PI 7). This divergence in perceptions of meaningful engagement underscores the importance for investigators to “listen and…have that two-way dialogue” (PI 7) to understand stakeholders’ interests and needs so they can be responsive and support engagement.
SHs described authentic engagement as "…a partnership. It’s a collaboration, that’s really what it is’ (SH B). Such a partnership entails being informed about the research environment beyond the study itself. “I think people would be more interested if they had a way to get some further information… For instance,… ‘how does the CTSI decide which of the projects get funded?’ … I think even community members would get some additional buy-in if they had…that type of information” (SH C). Being part of a collaborative team means being involved in shaping the project from the start. As a stakeholder noted, “…one really important piece is…having the questions come from the communities rather than…saying ‘This is what we want to study. Can you help us do it?’” (SH A). An authentic partnership values stakeholder expertise from the start and is mutually beneficial throughout.
The second barrier was building relationships with stakeholders, especially those most appropriate for a given study (2 PIs, 1 SH; Table 6). Limited networks and lack of access to the most appropriate stakeholders made it more difficult to successfully engage relevant communities and establish strong connections for investigators and stakeholders to effectively work together.
Two recommendations attempt to address this relationship barrier: one focusing on investigators and the other on stakeholders. For investigators, the recommendation is for funders and research support organizations to provide practical support, including training and facilitating stakeholder connections (10 PIs; Table 7). Suggested strategies involve “how to” training that includes guidelines for and examples of impactful CSE and CSE plans development, mentorship, and tailored feedback. Most PIs thought the primary responsibility for this training should be with funders and research support organizations, and one PI encouraged investigators to be proactive in learning CSE implementation methods early in their careers.
In addition to training, research support organizations can play a central role in developing and sustaining “pre-existing community relations…[with] organizations that would be easy to tap into” (PI 5). For example, holding events like matchmaking “lunches,” “round robin days,” or “some sort of date night” with investigators and stakeholders were suggested as valuable opportunities for investigators and stakeholders to meet and build rapport. This would directly mitigate the barrier of limited access to those stakeholders most relevant for a particular project. PIs also recommended three additional proactive strategies for investigators to pursue: networking (6 PIs), connecting with government or community organizations with interests related to the research topic (2 PIs), and thinking broadly about the research topic to include all potential stakeholders (1 PI).
For stakeholders, research support organizations and funders should prepare them to engage in research and demonstrate how it benefits them (4 SHs, 2 PIs). Better preparation could help clarify expectations, highlight the value of participation, and encourage commitment—even when other responsibilities compete for time. Research support organizations should be more “visible” by “getting word out there” (SH C) and providing broad information about research participation to stakeholders, particularly for those who “don’t know [it’s] there” and therefore cannot participate (SH D). This includes training stakeholders on their role in research and encouraging them to maximize the meaningfulness of engagement by approaching it with an open mind. In addition, research support organizations should demonstrate the benefits of engagement for SHs. These include “opportunities for leadership and growth” (PI 3), fostering trust, and enhancing stakeholders’ ability to “know what’s happening on the ground” (SH B) to inform their work in serving and advocating for their communities.
A third barrier, sustaining CSE, was vulnerable to limitations of stakeholder capacity and funding to support it (3 PIs, 1 SH; Table 6). Limited stakeholder availability and turnover at community organizations undermined consistent engagement. In some cases, SHs “did not come through” on tasks (PI 6) or did not have skills relevant to the study, such as speaking the language of the community. Additionally, finite funding to compensate and re-engage stakeholders in the latter parts of a study or in subsequent studies, also undercut consistent engagement.
Compounding these challenges of limits on capacity and funding within a study team, standard research practices added a fourth barrier to stakeholder participation (2 PIs; Table 6). Some requirements, such as documenting qualifications in an NIH biosketch format or completing the full Collaborative Institutional Training Initiative (CITI) certification, were perceived as “arbitrary barriers for community partners” (PI 3). These processes required time and effort, which was especially difficult for stakeholders already stretched thin by limited availability or organizational capacity, thereby slowing research progress. Moreover, long lag times for publication could affect the feasibility of re-engaging stakeholders around results and informing future directions for the research. Funding cycles may also not align with the timing of available results, leaving teams without resources to sustain collaboration.
These intertwined issues reflect respondents’ recommendations emphasizing the need for a culture change to fully implement the shifts in research practices and expectations to maximize participatory research (Table 7). From the perspective of PIs, funders are well positioned to drive this culture change by encouraging or requiring CSE. PIs underscored that funders should award sufficient resources to support CSE, such as protected time for investigators to cultivate networks and compensation for stakeholders (10 PIs). Investing in greater institutional and financial support could help create a more supportive research environment for long-term, meaningful CSE.
Discussion
The facilitating practices and recommendations identified across pilot projects offer practical strategies for investigators and stakeholders as well as a roadmap for funders and research organizations to strengthen CSE in preliminary, proof-of-concept health projects across the translational spectrum. Involving stakeholders in these short-term, modestly funded projects can strategically position follow-on studies for translational success. The strategies elucidate authentic engagement in this context and underscore the crucial role that research funders and research support organizations can play in catalyzing and supporting participatory pilot research.
Stakeholders described authentic engagement as a collaborative partnership in which they are informed about the research environment and help shape a specific project from the beginning. The disconnect we identified between PI and stakeholder views of what makes engagement meaningful for stakeholders highlights the importance for PIs to be intentional and curious about stakeholders’ experiences and needs so that they are able to support their involvement. Within the Broadly Engaged Team Science (BETS) framework, such relationships underscore the importance of integrating a wide range of perspectives and bidirectional learning as core team science competencies rather than optional practices. Stakeholders did not indicate differences in their view of authentic engagement across different aspects of a study, so investigators can work to operationalize the same vision of collaborative partnership in each aspect of the study lifecycle. Practical strategies for implementing authentic stakeholder engagement are summarized in an accompanying checklist (Appendix 1: Practical Strategies for Researchers Checklist).
Some of the facilitating strategies and recommendations for effective engagement align with what was previously known, while others are newly reported in the context of the initial pilot projects focused on here (Table 9). The short project timelines of these pilot projects may create immediacy that prompted additional practices or unique of gaps. Similarly, the challenge of scheduling and conducting team meetings in a format that is both convenient to encourage attendance and conducive to team building, which often arose due to CSE necessitating larger groups, was newly reported in this context. The need to align calendars and meet long enough to make decisions on a short timeline may have exacerbated these already challenging aspects of team science. The other barriers that emerged from these participatory pilot projects aligned with existing literature about longer-term projects.
Of particular note, investigators and stakeholders engaged in the pilot projects emphasized the need for culture change to support effective CSE, going beyond the practices of individual investigators and stakeholders. Their perspectives highlighted the role of research funders and research support organizations in promoting and facilitating CSE within translational research. These organizations, including but not limited to Tufts CTSI, are well positioned to encourage or require CSE and can provide the needed funding and other resources to support it. For example, as funders, CTSA hubs across the U.S. can require and provide adequate funding for CSE in sponsored research. As research support organizations, CTSA hubs can also provide investigators and stakeholders with resources, training, and guidance to prepare for and sustain CSE, particularly in contexts where formal stakeholder engagement processes remain limited or inconsistently defined and where proposal and post-award requirements are often designed for academic researchers rather than community participants. (LeClair et al., 2020; Seidel, 2022; Selker & Wilkins, 2017).
Organizations with fewer resources can adapt many of the strategies and recommendations, even if on a targeted basis. The underlying practices and policies—facilitating connections, building investigator and stakeholder capacity, and protecting time and resources for engagement—are broadly applicable and could be achieved through shared toolkits, regional partnerships, or cross-project mentoring. Effective adaptation and implementation of engagement resources depends on aligning engagement investments with organizational priorities, investigator expertise, and the translational stage of supported projects. For example, organizations supporting a higher proportion of later-stage (T2–T4) studies or community-facing research may benefit from investing in systems that operationalize and sustain engagement—such as coordinating ongoing partnerships, supporting implementation activities, and providing access to engagement expertise. Alternatively, organizations with more researchers or projects focused on early-stage (T1–T1.5) development may prioritize strategies that establish engagement foundations, including identifying appropriate stakeholders, building initial relationships, and incorporating stakeholder perspectives into study design.
Learnings in this report bolster and specify aspects of the Broadly Engaged Team Science (BETS) framework. BETS emphasizes fit-for-purpose engagement, the integration of stakeholders as authentic and integral members of the research team, and collaboration shaped by context and translational stage (Selker & Wilkins, 2017). These principles are feasible to implement even in projects that are short in duration with minimal funding. In addition, engagement is also feasible in projects at earlier stages of translation, focused on initial development and feasibility testing. Seven out of the nine PIs leading projects at early translational stages had engaged stakeholders, even though CSE may appear more challenging in early-stage research.
Actionable strategies at two levels—individual and organizational–concretize the BETS framework. At the individual level, participants described practices that establish engagement, including leveraging existing relationships, aligning research aims with stakeholder priorities, and incorporating stakeholder perspectives into study design. At the organizational level, participants emphasized supports that help initiate and sustain engagement, such as facilitating connections between investigators and stakeholders, providing access to engagement expertise, and ensuring clear and consistent communication to facilitate ongoing collaboration.
These accounts underscore the importance of organizational-level investment–highlighting that successful implementation of broadly engaged team science depends on funders, research support organizations, and translational research infrastructure, including CTSAs, to help enable and sustain stakeholder engagement. Such entities play a critical role in navigating practical requirements associated with stakeholder involvement —such as budgeting, documentation, and training—that shape how stakeholders can meaningfully participate in research (Seidel, 2022; Selker & Wilkins, 2017) (Figure 2).
Considerations for interpretation
Confidence in our findings reported here is bolstered in two mutually reinforcing ways. The in-depth nature of qualitative interviews allowed PIs and stakeholders the flexibility to describe their experiences, which allowed some previously unreported findings to emerge in the context of CSE in small initial studies. These findings are situated alongside themes consistent with those in the academic literature, supporting face validity.
At the same time, several limitations should be considered when interpreting our findings. The SH perspectives we report in particular should be interpreted as illustrative rather than a comprehensive portrayal of all stakeholders engaged in pilot projects.
-
Saturation and representation: SHs for this study were identified through PI nomination because a centralized database of SHs engaged in the pilot studies does not exist and follow-up contact information for SHs was not always available. As a result, fewer SHs were recruited than desired and thematic saturation of SH perspectives is uncertain. In addition, reliance on PI nomination introduced a gatekeeping power dynamic to SH recruitment that may have influenced which SH perspectives were included.
-
Limits on assessing representativeness: While some information on SH characteristics was available for all 14 pilot projects — from either SHs themselves and/or PIs — detailed demographic characteristics and positionality were not systematically revealed by SHs. Interviewed SHs were primarily organizational representatives or clinicians. While they offered valuable insights informed by their professional and organizational experience, the lived experience of community members most affected by health conditions may not have been fully captured.
In addition, further validation and replication are needed to ensure generalizability.
-
Sample size: The relatively small number of projects and participants limits generalizability, and it is possible that interview participants may differ from nonparticipants (e.g., having more positive, or more extreme, engagement experiences). Accordingly, this report avoids causal language, and additional validation and replication with other small-scale pilot projects across the translational spectrum are warranted.
-
Pandemic context: Engagement practices may have been influenced by COVID-19–related disruptions, as two of the three cohorts of pilot projects were at least partially conducted during the pandemic. Several factors, though, suggest any effect on the pilot project practices and strategies reported here was smaller than might be expected. 1) Although some teams received extensions during the pandemic due to research slowdowns, the scope of each pilot study remained consistent with what could be accomplished within one year of typical research progress. 2) Among the 24 eligible projects, there were no differences between those represented and not represented in time to expend funds (with or without extensions) or in pandemic-related delays. Median award amounts differed slightly ($45,000 vs. $30,000), but both fall within the range of modestly funded projects. 3) Analysts minimized potential bias by focusing on themes not dependent on pandemic conditions to improve generalizability.
Finally, this report is unable to address two important issues: 1) SH compensation was not systematically assessed, and 2) this report about engagement practices and strategies was not designed to evaluate the impact of engagement on pilot study outcomes or subsequent larger-scale research.
Conclusion
This article elucidates facilitating practices, barriers, and recommended strategies to strengthen participatory research within modestly funded, short-term pilot studies that lay the groundwork for larger clinical research. Lessons learned underscore the importance of pairing project-level practices with organizational supports to enable and sustain authentic engagement. This points to a shared responsibility: funders and research support organizations (e.g., CTSA hubs) are well positioned to strengthen the culture and infrastructure needed for widespread participatory research, while investigators partner with stakeholders to actualize it. Strategically implementing these lessons for initial small-scale projects could help shape more responsive and relevant health research across the translational spectrum.
Funding Statement
This project was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, Award Numbers UL1TR002544 and UM1TR004398. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Advancing Translational Sciences or the National Institutes of Health.
Competing interests
The authors have no conflicts of interest to declare.
Author Contributions
The authors confirm that Thomas W. Concannon, Robert D. Sege, Alice Rushforth, Aviva Must, Nadia Prokofieva and Lisa C. Welch contributed to conception and design of the study. Sarah K. Brewer and Lisa C. Welch were involved in data collection. Sarah K. Brewer and Siyu Chen analyzed the data. Sarah K. Brewer, Siyu Chen, and Lisa C. Welch discussed and contributed to interpretation of results. Sarah K. Brewer, Siyu Chen, and Lisa C. Welch drafted the manuscript. All authors contributed to manuscript revisions, approved the submitted version, and agreed to be accountable for the work.
Acknowledgements
The authors would like to thank Dennis Sunder for his contributions to data collection and coding.
Data availability statement
Deidentified data are available upon reasonable request.
Corresponding Author
Siyu Chen, MPH
Tufts Clinical and Translational Science Institute
35 Kneeland Street, 7th–11th Floors
Boston, MA, USA.
Email: Siyu.Chen@tuftsmedicine.org.