Participatory Sensemaking Through Data Parties: Lessons From Cross-Case Research and Evaluation

Participatory processes are increasingly common in social sciences research and evaluation practice. Program evaluation has evolved beyond top-down judgment toward a more formalized and responsive transdiscipline that can respect diversity, improve quality, and promote the usefulness of programs in our social world (Cousins, 2020; Scriven, 2008; M. F. Smith & Mathison, 2005). This renewed focus on evaluation use creates space for methodological innovations that respond to the shifting nature of programs, policies, and social contexts (Patton, 2010; Thomas & Campbell, 2020). As evaluators continue to reimagine how evaluation can be conceptualized, designed, and implemented, research on evaluation plays an important role in strengthening the evidence base for rigorous, relevant, and useful practice across diverse settings (Alkin, 2012; Coryn et al., 2016; Mertens & Wilson, 2018). This paper examines how data parties function as a participatory sensemaking practice across three educational evaluation and research contexts. After introducing the authors and vignette contexts, we outline the conceptual and methodological foundations of the inquiry, present three vignettes illustrating how data parties were designed and facilitated, and then offer cross-vignette reflections and practical recommendations for future use.

A data party is an interactive, time-limited event that brings together a range of interest-holders, such as program champions, designers, staff, participants, and community members, to collectively engage with and interpret data (Rogers & Newhouse, 2020). Data parties are designed to foster inclusive sense-making by making data accessible, encouraging open dialogue, and facilitating the co-construction of meaning (Lewis et al., 2019). Consistent with broader interest in participatory methods for research evaluation (e.g., Franz, 2013; Sperling et al., 2025), data parties represent an interactive engagement approach to sharing and discussing data with interest-holders directly. For example, during evaluations in education, researchers and evaluators may convene a data party with administrators, teachers, students, and families/caregivers to explore emerging findings. By involving a range of people and perspectives in the analysis, data parties can support nuanced understanding that allows participants to affirm, question, or complicate interpretations, and contribute to actionable conclusions (Hutchinson, 2017).

Beyond supporting analysis, data parties serve relational and pedagogical functions found in sensemaking. Klein and colleagues (2006) explain that sensemaking is “a motivated, continuous effort to understand connections (which can be among people, places, and events) in order to anticipate their trajectories and act effectively” (p. 71; see also Dervin, 2003). Sensemaking in data parties is an action-oriented process where people integrate experiences with data, looking for patterns and making connections. Through intentional processes, including structured agendas, data displays and prompts for dialogue, data parties can promote shared learning. These processes deepen trust and contribute to relationships by helping to break down hierarchies and create a welcoming environment for collective inquiry (Kolko, 2010).

Collective inquiry involves a range of perspectives that can positively influence evaluation by promoting shared ownership of data, building an understanding of evaluation purpose, and providing a process for evidence-informed decision making (Chouinard, 2013; Cousins, 2020). As a participatory sense-making tool, data parties exemplify the values of inclusive and equitable evaluation by making the analytic process more transparent and accessible (Lewis et al., 2019). In this way, data parties operationalize the principles of participatory processes, offering a mechanism through which joint analysis and the co-production of knowledge can occur (Ligtermoet et al., 2025; Rogers & Newhouse, 2020).

This paper is structured to provide an overview of the literature informing the effective use of data parties, providing the context in which data parties were used for the inquiry (i.e., three vignettes), a detailed reporting of what occurred during data parties, and the findings that resulted from their use. While the inquiry has several layers of data collection, analysis, and sense-making, this paper offers methodological rigour to support best practices for future uses of data parties in engaged contexts. First, we provide author positionalities in connection to the vignettes where data parties were used. Then, we provide the conceptual foundation of evaluation connected to data parties. Next, we provide a detailed description of the methodological approach to preparing for and implementing data parties across case sites and approaches to data analysis. To support cohesion, case sites are first presented alongside the others, but are detailed individually to ensure contextual specificity is represented and explored in depth. Finally, data is integrated to crystallize findings (Denzin & Lincoln, 2011) and provide guidance and implications towards future research applications.

Authors and Vignette Contexts

We are a team of six author-participants. Each of us holds advanced degrees in education with a range of experiences in K-12 education, evaluation, and research. As practitioners who have applied the data party technique, we conducted this case study to understand the technique more deeply and contribute to empirical literature about data parties. The three data parties were led by different author combinations and conducted with various interest-holders during their respective evaluations. Data sources we engaged in each vignette also varied. Far from creating an apples-to-oranges comparability problem, we contend that these variations reflect the responsive, participatory nature of collaborative evaluation work (Cousins, 2020; Shulha et al., 2016). Throughout this paper, we adopt the term interest-holder as a deliberately inclusive alternative to stakeholder, better reflecting those with legitimate interests (Akl et al., 2024) and supporting participatory practices from the foundations of program evaluation. We use this term intentionally because it better reflects the relational and participatory commitments of this work. Rather than positioning people as peripheral “stakeholders,” interest-holder emphasizes that those involved have legitimate, situated interests in the process, interpretation, and use of findings. This shift is especially important in participatory research and evaluation, where people are engaged as contributors to meaning-making rather than as passive recipients of results.

Establishing a Broad Conceptual Foundation in Evaluation

Participatory and collaborative processes are two important foundations within the range of approaches, frameworks, and methods that mark a shift from traditional evaluation toward more people-centred, context-aware, and learning-oriented practice that supports meaningful and lasting results (Teitelbaum, 2020). The field of evaluation has developed a range of conceptual approaches, including participatory evaluation (Cousins & Whitmore, 1998), empowerment evaluation (Fetterman, 2001), utilization-focused evaluation (Patton, 2008), developmental evaluation (Patton, 2010), culturally responsive evaluation (Hood et al., 2014), collaborative approaches to evaluation (Shulha et al., 2016), and transformative evaluation (Mertens, 2021). Despite their variations, these foundations collectively recognize that evaluation is meaningful when it centers and reflects the insights of those most intimately connected to programs—designers, implementers, participants, and affected communities (Lewis et al., 2019). These approaches, while distinct, share a commitment to contextually grounded, interest-holder-engaged practices aligned with professional standards and competencies (e.g., American Evaluation Association, 2018; Canadian Evaluation Society, 2018; Joint Committee on Standards for Educational Evaluation, 2010).

Participatory approaches to analysis can contribute to the co-construction of understanding while prioritizing community voices at key junctures of evaluation. Fine and colleagues (2021) suggest that participatory analytical processes are grounded in the belief that needs are socially constructed, that analysis benefits from diverse perspectives, and that voices often marginalized in the social sciences should be present in knowledge production. This aligns well with other participatory research methods (such as PAR) that aim to address social problems (Gilbert, 2022). Organizations that engage in formal research are beginning to explore ways to conduct participatory data analysis through formalized evaluation processes (Community and Stakeholder Engagement Program & North Carolina Translational and Clinical Sciences, 2023; Tessera, 2022), and emerging empirical and evaluation studies have begun using data parties as structured participatory approaches for engaging interest holders in data interpretation and meaning-making (Anyolitho et al., 2024; Bird & Lewis, 2021).

With its broad conceptual foundation, evaluation is well positioned to support participatory processes aimed at social change by considering culture, context, and multiple interpretations when asking ‘what counts’ as credible evidence. Including the perspectives of marginalized communities can deepen and enrich understandings of credible evaluation (Mertens, 2016). More broadly, centering interest-holder perspectives can support more inclusive knowledge production and enhance the credibility, relevance, and practical usefulness of findings (Cousins & Whitmore, 1998; Mertens, 2021). This paper contributes to the growing literature on participatory analysis by examining how knowledge production can be expanded through the inclusion of voices that are often absent from scholarly interpretation. Beyond conceptual framing, it also highlights the need for greater clarity about how the co-development of understanding can be meaningfully integrated into evaluation practice in ways that are responsive to pragmatic needs.

The three vignettes in this paper used data parties to engage interest-holders as active contributors in evaluative processes. Hosting data parties reflects the ethos of Collaborative Approaches to Evaluation (CAE), which emphasizes shared knowledge development through participatory, relationship-centered practice (Cousins, 2020; Shulha et al., 2016). Empirical studies highlight CAE’s effectiveness in enhancing both the quality of evaluation findings and the learning it generates across individual and organizational levels (Acree, 2019; Cousins, 2020). Drawing on three educational evaluation and research contexts in Canada, this paper examines how data parties function as a participatory practice for collaborative sensemaking.

Methodology

Our method, like the data parties themselves, is multifaceted, bringing together different aspects of social science research. We align with pragmatic and critical postpositivist perspectives, encouraging multiple theories and methods as a necessary part of complex human phenomena associated with evaluation and evaluation research (Shadish, 1993; Tanlaka et al., 2019). To start, we came together as evaluators and researchers engaged in participatory processes, interested in negotiating meaning with others, and valuing richness brought from plurality of perspectives and experiences. Figure 1 illustrates how we conceptualized data parties as participatory approaches to collaboration across three distinct vignette contexts. Vignette 1, a youth-engaged mental health project, was facilitated by two authors, while Vignettes 2 and 3, which involved a community-engaged higher education initiative and a study of K–12 teacher assessment identity with technology, were facilitated by three authors. Together, these vignettes show how data parties supported collaborative sensemaking across varied evaluative and research settings.

Figure 1.Participatory Data Parties Across Three Vignette Contexts

We initially framed our process as descriptive case study research involving the in-depth examination of a small number of cases where data parties were used as part of the process (Merriam, 1988; Stake, 2013). Quickly we realized that our shared efforts diverged from case-based research by recognizing the value of collaborative ecologies as a way of generating knowledge (Mulvihill & Swaminathan, 2023). Like the sensemaking processes we engaged in within the vignettes, our method grew to be “active meaning-making through dialogue” and artefact review to arrive at new knowledge (Paulus et al., 2008, p. 231). These processes are akin to collaborative autoethnography that enable our dual roles as both contributors and investigators (Hernandez et al., 2017; Phillips et al., 2022). Our approach descriptively narrates our processes to examine cycles of planning, implementation, and reflection. We begin by outlining the authors and vignette contexts (Figure 1), detailing our data sources (Table 1), and sharing analysis processes to make visible the people and processes of this research.

These CAE projects were undertaken during a global pandemic and were restricted to online gatherings. Thus, all data parties took place virtually using various online engagement tools (e.g., Nearpod, Kahoot, Padlet, Jamboard). Materials were not shared in advance, and no follow-up tasks were assigned after the sessions, ensuring that all insights were generated during the data parties themselves. The post-event dialogues included both informal conversations among authors, as well as formal discussions during our shared sense-making and analytic processes.

Data Analysis

Consistent with the CAE framework (Cousins, 2020; Shulha et al., 2016), our collaborative efforts extended from data collection into the data parties and subsequent analysis. This required ongoing relationship-building and iteration between describing, generating, and reflecting on data as well as developing new understandings across each case (Fiddler, 2021). In post-event dialogues, we highlighted key analytical moments that emerged both during research conversations and later reflective moments (Tobin, 2014). Inspired by Fiddler’s (2021) concept of indexing research conversations, we developed a six-step process (informal collaborative discussions, structured show-and-tell session, individual and paired analysis of artifacts, video-recorded read-aloud session, individual reflection on transcript, and collaborative thematic synthesis), represented in Table 2, that aligned with our group’s tendency to build camaraderie, learn from one another’s experiences, and make sense of research through conversation.

Throughout this multi-step process, analysis was approached through a lens of reflexivity, with individual experiences and insights considered as valuable data contributing to understanding. After the vignettes to describe the cases and data parties, we offer a cross-case reflection of our shared learning.

Data Party Vignettes

Vignettes are descriptive instances of specific scenarios intended to simulate real-world cases presented in written or visual formats (Skilling & Stylianides, 2020). They can be useful in enhancing researchers’ personal understanding of research processes (Barnes, 2018), and representational richness and reflexivity in autoethnographic contexts (Humphreys, 2005). We detail the three cases as vignettes to illustrate data parties as a participatory process across different contexts, each with its own goals, participants, and outcomes. Each vignette shows how data parties involve a structured yet flexible approach that encourages joint analysis and knowledge sharing among diverse interest-holders (Lewis et al., 2019; Rogers & Newhouse, 2020). Within these vignettes, we provide the context, purpose, and process to detail the inquiry with each situation. We intentionally outline timepoints that describe when data parties were conducted, and a rationale about how the timepoints are contributing to the inquiry. The structure of each vignette provides details that frame the pragmatic use of data parties as a participatory analysis, and models to ground thinking about potential use cases for data parties beyond evaluation in formalized empirical research contexts.

Vignette 1. Youth Engaged School District Mental Health Project

Context

The importance of prevention and early intervention makes schools an ideal place to provide mental health and well-being education and support to children and youth (School Mental Health Ontario, 2024). In Ontario, Canada, where schools are a provincial mandate, mental health is considered a top priority that contributes to a foundation for student learning. Funds have therefore been allocated to implement and evaluate a tiered approach to mental health (Ontario Ministry of Education, 2022). We have been engaged in a multi-year CAE focused on the effectiveness of youth mental health and well-being in school districts in southwestern Ontario. The CAE takes place in an English-language school district that offers publicly funded, faith-based education across elementary and secondary schools.

Purpose

During 2020-2021, this CAE focused on a district-wide mental health and well-being project offered in conjunction with a community health organization with a shared priority focused on youth engagement. The project team included 16 people. It was led by a superintendent who served as the program champion, a district mental health lead, three members from a community health organization, and two evaluation researchers assisted by four university research assistants. In addition, as part of the youth engagement commitment, five high school students provided leadership on the project team.

Process

The project team met every 4-6 weeks to discuss the overarching CAE focused on youth mental health and well-being in schools. Additionally, student leaders worked with evaluators monthly to advance the youth engagement initiatives focused on student-led Tier 1 mental health awareness and support. These meetings included two data parties: (1) one with data focused on system mental health initiatives that included case load data and a calendar of mental health activities (e.g. professional learning, community engagement, clinical note taking) and (2) a second data party where the project team and additional district and youth leaders examined data from approximately 300 participants (grade 6-12 students and teachers) who had attended a youth-led, full-day, virtual event about mental health and well-being.

Data parties lasted 90 minutes and were held through Zoom. Prior to the data parties, we reviewed, organized, and selected data that aligned with the established questions, showcased a range of participant voices, provided different methodological snapshots of initiatives, and would be interesting to the data party attendees. For example, rather than sharing an excel file with reams of raw data, we parsed the data into more bite-sized pieces that could be accessed and discussed in an online setting. Although the data was prepared prior to the meeting, we intentionally did not share information or resources in advance, as data security was a concern. We wanted to ensure that participants could work with the data in a session with us but limit the mobility of the data prior to completing the analysis and reporting.

Owing to the large amount of data, interest-holder diversity, and limited time for each data party, the agendas for both data parties followed this format:

• Introductions and welcome

• Stated inquiry questions for the CAE

• Overview of methods/data to be examined today

• A look at big picture data - quantitative responses

• Digging deeper with qualitative data

• Thoughts and questions

Timepoints

Data parties occurred at two timepoints throughout this inquiry; the first occurring in the early stages of the evaluation while the second occurred near the end of the evaluation to appropriately integrate data sources and engage key interest-holders in collaborative interpretation.

Timepoint 1. The first data party was held in the early stage of implementing the evaluation and was designed to serve multiple purposes. The first purpose was to build relationships across the project team while establishing credibility and utility of the CAE processes. Since some team members were new to collaborating with us, we used the data party process to show that we valued and invested time in their expertise. As a mixed and multiple method CAE, we wanted to demonstrate how different data sources could be integrated to support a holistic response to inquiry questions. At this stage, we wanted interest-holders to have visible and tangible evidence of the inquiry progress as well as a chance for shared reflective conversations about ways in which the data was contributing to the broader inquiry goals. A core goal of the first data party was to develop capacity with the core project team before enlarging the group to include additional district and student leaders as data party participants.

Timepoint 2. The second data party enabled us to engage in data sharing while also including district and youth leaders, including students from local high schools. To promote engagement, we included icebreakers, dialogue, and various structured activities, such as responding to reflection questions on Padlet and NearPod. For example, we began with a teambuilding reflective activity to offer one word to describe the youth engagement experience. Words offered by participants included: awesome, awe-inspiring, cool, ameliorative, knowledge-filled, rad, excellent, magical, knowledgeable. Reflecting on qualitative data was more structured than the first timepoint. In timepoint two, we invited participants to use a Padlet to tell us what they were thinking, feeling, hearing, seeing, and wondering. Similarly, structure was added when participants reviewed exit data and gave us recommendations for moving forward. As researcher-evaluators who are also educators, we used our pedagogical perspective and insights to design a lively data party with opportunities for youth and adult engagement.

Vignette 2. Evaluation of Community-Engaged Initiative in Higher Education

Context

This evaluation was conducted by three novice evaluators under the guidance of an experienced evaluator. Over the course of one year, we learned about evaluation by conducting a CAE with university-based program managers. The program was a community-engaged extracurricular project situated within a higher education institution. In that program, interdisciplinary teams of PhD students with mentor support responded to applied research needs identified by a community partner. The program is aimed at upper year doctoral students with teams working from the Fall until a Spring celebration.

Purpose

Program managers had positive feedback from numerous participants and commissioned the evaluation to learn how the program could grow. The CAE was established to respond to three main areas of interest: positioning the initiative as a signature program at the university; exploring ways to emphasize equity, diversity, and inclusion; and to identify strategies to streamline the administrative load. Throughout the evaluation, we worked with two program managers who were the primary interest-holders.

Process

After discussing the strategies and techniques, we agreed to include a data party to collect interest-holder feedback, add additional perspectives, and further contextualize our findings. This data party also served as an opportunity to introduce some difficult student feedback, reflect with these interest-holders on that feedback, and use those insights to inform the development of our final evaluation report.

At our primary interest-holder request, we also hosted an additional data party, which took place prior to the initially planned data party because of the timing and access to program participants. While this data party emerged as the evaluation itself was unfolding (King & Stevahn, 2012), the insights students provided proved critical to the evaluation findings. Students are also recognized as interest holders in this program because they have an interest in the processes and outcomes of an effort; they have dedicated their time and energy towards a yearlong co-curricular experience. For example, student participants provided in-depth insights into the differences between how participating and non-participating doctoral students perceived the program – a key focus for the primary interest-holder in understanding both communities.

The data parties occurred after the bulk of data collection was complete, however they themselves served as data collection opportunities. Descriptions of the data parties are below, first the unplanned experience with students and then one with the primary interest-holders.

Timepoints

Data parties occurred at two timepoints with different interest-holders throughout this vignette; the first occurring with students after the experiences with some interest-holders (i.e., students) and the second occurred after primary data collection had concluded with other interest-holders (i.e., clients).

Timepoint 1: Student Debrief Party. We had not interacted with the current program students when we planned our session, so we prepared for a modest conversation with activities so participants could submit their thoughts verbally or in writing during the session. Guided by salient data in the survey responses and our client’s interests, we prepared the following agenda:

1. Introductory activity

2. Reflecting on program highlights

3. Program strengths / challenges

4. Three large group conversations in response to survey data

5. Comparing the program to other doctoral experiences

6. Participant advice for future iterations and next steps

7. Two open-ended activities with reflective prompts

The participants had a lot to share, and we quickly strayed from our timeline. It was exciting for the evaluation team to see such enthusiasm and diverse ideas from the students. By typing collaborative real-time notes in a separate document, the evaluators rearranged the session’s agenda as the discussion unfolded. We found that some later activities were not needed because the conversation found its way naturally to desired topics.

Timepoint 2: Client Data Party. When we were planning the second data party, our data collection was complete, and we had some clear ideas that we felt needed to be addressed in the report. This data party also included a newly hired interest-holder who knew little about the program, much less our evaluation. This is common in research and evaluation partnerships: since evaluations can occur over long timespans, those involved can change, necessitating new relationship building efforts as part of the CAE approach (Shulha et al., 2016). To facilitate reflection and discussion with the client team, we prepared the following agenda:

1. Welcome and rapport-building

2. Appreciative reflection

3. Five large-group discussions based on intriguing student survey data

4. Two open-ended reflective activities about signature programs

5. One nested analysis activity (read one page of data, reply to six prompts, create a ‘found poem’, see Wiggins, 2011)

6. One nested categorization activity (read one page of recommendations, individually prioritize them, large group discussion)

7. Two open-ended reflective activities about the data and the data party

8. One group discussion about next steps in the evaluation

These activities seemed to support new ideas and ‘aha!’ moments (Kounios & Beeman, 2015) because at some times, the interest-holders stopped typing their thoughts and spoke out loud with each other. In response, the evaluation team relied heavily on our shared note-taking documents to capture their ponderings before they were forgotten.

Vignette 3. Evaluating K-12 Teacher Assessment Identity with Technology

Context

This data party was conducted as part of a master’s thesis focused on analyzing the assessment identity of K-12 teachers integrating technology into their assessment practices. The research aimed to explore how teachers’ identities as assessors are influenced by technology use in their practice. This data party was a purely post-hoc analysis designed to provide additional insights into the original research process and its subsequent findings. This collaborative analysis not only helped refine the interpretations of the original study but also generated insights for future research and evaluation in similar contexts. To enhance the depth and trustworthiness of the findings, graduate researchers were engaged in this post-hoc collaborative analysis. Graduate students with varied research interests and methodological skills were recruited, none of whom had any prior involvement with the case or its data.

Purpose

The primary goal of this data party was to add new voices to the analytic process and, in doing so, strengthen the value of the thesis findings. Working collaboratively, the graduate researchers aimed to (1) discuss the original interpretations, (2) surface additional patterns, tensions, or unanswered questions that a single-author analysis might miss, and (3) translate these insights into concrete recommendations for future scholarship and for teachers integrating technology into their assessment practice.

Process

The data party was held in two phases. The first session took place over two hours, during which graduate researchers participated in blind coding of a subset of the data using an inductive, exploratory approach. The session followed a structured agenda based on the Accelerated Learning Cycle (ALC) framework (A. Smith, 1998). The ALC model supports a collaborative learning environment that acknowledges the participants’ diverse knowledge and work styles, ensuring an organized and purposeful session. The ALC process unfolded in four deliberate stages: welcoming and purpose-setting; an overview of qualitative coding followed by a collaborative coding exercise; independent inductive coding of a shared data sample; and a group intercoder-reliability check to reconcile codes and disagreements. Following the first session, the same graduate student participants were then invited back for the second timepoint, a virtual gallery and discussion session.

Timepoints

Data parties occurred at two timepoints throughout this vignette; the first occurring after initial data collection for the first phase of analysis while the second occurred in the second phase of analysis for collaborative interpretation.

Timepoint 1. During the first session, graduate participants collaboratively coded 10% (n=17) of the discussion post data to establish intercoder reliability (ICR). ICR is used to measure the consistency of coding between multiple coders and supports the trustworthiness of qualitative research findings (Saldaña, 2021). A random number generator was used to select discussion posts, ensuring an even distribution across the five weeks of data, with three posts selected per week and two randomly chosen from the two weeks with the most posts. While there is no universally accepted agreement percentage, a minimum benchmark of 85%-90% is generally considered acceptable (Saldaña, 2021). To reduce bias, participants did not have access to a predefined code list, as this allowed for an inductive coding approach where participants could generate codes based solely on the data, minimizing the influence of pre-existing assumptions. Afterward, the researcher and participants reviewed the codes, discussing agreements and disagreements to reach consensus. The ICR was calculated by dividing the number of agreements by the total codes, ultimately reaching an agreement rate of 93.6% (320 codes, 309 agreements, 21 disagreements).

Timepoint 2. After Timepoint 1, the coded data was reviewed and a preliminary code list was developed, which was used during the second phase of the analysis. For Timepoint 2, the same graduate student participants were invited to a virtual gallery and discussion session. This session involved presenting the study’s findings and inviting participants to explore the data in a visual format (in this case, a Google slide deck). The slides included tables summarizing the codes, descriptions, and sample quotes. Guiding questions included: What stands out to you in the findings? Are there any surprises? Do the results align with the original research questions? Participants explored the slides, discussed the findings, asked questions, and shared concerns. This session allowed for collaborative analysis and refinement of the data interpretations, contributing to a deeper understanding of the findings and their implications for practice.

Making Sense through Cross-Vignette Reflections

A cross-vignette exploration highlights three key themes related to optimizing data parties as a participatory practice in research: (1) prioritizing relationships, (2) putting CAE principles into practice, and (3) structuring data for collaborative engagement. Sensemaking as a process for engaging in reflexive evaluative inquiry was an integral component across vignettes, retaining a fluid and dynamic nature for relationships and engagement throughout the inquiry process. This was apparent through the various layers involved in this inquiry, observed (and experienced) through the diverse interactions and engagements within the data parties themselves. The following section contextualizes sensemaking in cases and in harmony with data parties as a methodological choice. Across the vignettes, these patterns suggest that data parties offer a theoretically meaningful model of collaborative evaluation that repositions sensemaking as an ongoing relational process rather than a final analytic stage.

Prioritizing Relationships

The uncertainty, emotional strain, and shifting priorities of the COVID-19 pandemic placed added pressure on the already complex work of collaborative research and evaluation. Those challenges also served as a reminder that researchers and participants alike are always navigating personal, professional, and social demands—regardless of the external context. Even in so-called ‘normal’ times, people bring their full lives into the research and evaluation process. Recognizing this, it becomes clear that centering and sustaining relationships isn’t just a crisis response; it is an ethical and practical responsibility. Building trust, showing care, and maintaining open communication are foundational to meaningful collaboration and should remain central to evaluative practice beyond moments of disruption.

In Vignette 1 (i.e., Mental Health Project), the researcher-evaluators had prior relationships with the core interest-holders which enabled trusting relationships and a mutual respect for working towards the established project goals. The initial relationships had been formed over a sustained period with multiple forms of communications and interactions. Yet, the researcher-evaluators had no prior relationships with the students and no access to the student environment. This necessitated an approach that prioritized building relationships and recognized that those relationships would develop over time through flexible and responsive communications.

In Vignette 3 (i.e., K-12 Teacher Assessment Identity), graduate students were recruited by the researcher based on diverse research interests and methodological expertise. Although the researcher had existing relationships with each of the graduate student participants, the full group met for the first time during the first data party session. Qualitative coding was new to many attendees, so building relationships and trust amongst the participants was important for developing a sense of community and fostering a safe space that allowed for open dialogue and creativity. Icebreaker activities were used to help participants get to know one another’s academic research interests and provided informal opportunities to share stories that provoked deep connections and conversation.

Bringing Participatory Methodologies to Life

Data parties, as a collaborative evaluation practice, do more than align with the CAE framework—they actively embody its principles. The CAE is one of many program evaluation approaches where participatory sense making processes embrace pluralistic ways of being in evaluation and research. It prioritizes meaningful relationships and working towards a shared goal. Part of developing and sustaining collaborative relationships is about finding ways for interest-holders to be involved in different aspects of the evaluation. CAE processes can be challenging, especially when jointly hosting analytic dialogues where values and interpretations are coupled with the breadth of data collected and the skill required for working with mixed and multiple methods. Yet, using a CAE requires a commitment to engage in appropriate participatory processes that offer opportunities to develop evaluative thinking and data parties are a possibility for these processes.

Given the different context described in these vignettes, CAE occurred in different ways. In Vignette 1 (Mental Health Project), the team wanted to recruit widely and broadly to engage youth. Vignette 2 (i.e., Community Engagement in Higher Education), for example, included the real-time rearranging of agendas in response to emerging interest-holder interests and questions. Given the context of the pandemic, they accepted that they needed to rely on the project team to help ‘tap shoulders’ of students who could get involved and then find ways to creatively encourage their commitments. Additionally, the researcher-evaluators needed to adjust their expectations for how many students would be involved. While the Vignette 2 facilitators came prepared to each session, meetings always included back-channel conversations in Google documents that allowed for quick pivoting and adaptability. Such ongoing communication and flexibility are essential in bringing a CAE to life.

Partitioning the Data

Each of our projects generated more data than we could share in a single data party. Therefore, each facilitation team needed to be selective about which data to share in a format that would be useful. Each of the facilitators reflected on the data individually and through conversations with colleagues and identified different strategies to select data samples for the party. To avoid selection bias, the Vignette 3 (K-12 Assessment Teacher Identity) researcher used a random number generator to identify 10% of the data set to be shared in the data party. Conversely, the Vignette 2 facilitation team hand-selected data and deliberately designed a prioritization activity with client interest-holders to stimulate planning discussions within their team. Finally, the facilitators of Vignette 1 curated data that seemed most accessible to a wide range of interest-holders and appropriately captured the project in its entirety. By comparing our strategies in creating the data party samples, we find that project purpose, analysis goals, context, and audience needs drive decisions involved in selecting sample data.

Discussing Data Parties as Participatory Processes

A great deal of intentionality is required to meaningfully engage in a data party within the context of research and evaluation. Across the three vignettes explored in this study, we found that building strong relationships and designing effective processes, such as data parties, can significantly enhance the accessibility, relevance, and practical utility of evaluation. These collaborative events are not incidental add-ons; rather, they require the same level of strategic planning, clarity of purpose, and attention to context that supports success in broader research and evaluation efforts. In practice, evaluators often adopt a blended framework or take an eclectic approach by drawing elements from multiple conceptual frameworks or methodological orientations to meet specific needs of a context or inquiry (Bledsoe & Graham, 2005; Christie, 2003; Hargreaves, 2021). Innovative methods, such as data parties, work well when used with evaluation and research approaches that are intentionally flexible, offering guiding principles rather than rigid or formulaic processes.

Our findings suggest that the impact of data parties often exceeded expectations. While initially intended as participatory analysis sessions, they became much more: spaces for reflection, relationship-building, and shared learning. Data parties function as a means of co-producing evaluative knowledge that enables evaluators to innovate and respond in real time—respecting the uniqueness and perspectives of multiple interest-holders. As prior literature has shown, data parties can help surface core issues (Gerken et al., 2016), promote deeper thought and inquiry (Hadar & Brody, 2013), support professional learning and skill development (Shagrir, 2017), and provide opportunities for reflective analysis (Santagata & Guarino, 2012). These layered outcomes show how data parties can enrich both the process and the findings of evaluation.

We discovered a contribution of data parties fostering a culture of evaluative thinking that is focused on bringing people together to advance learning functions of evaluation and advance collective understanding. To leverage learning through participatory processes, collaboration must be deliberately cultivated over time and intentionally crafted. Participants need multimodal opportunities to engage fully in the sense-making processes and contribute their lived experience, contextual insights, and critical perspectives (Gokiert et al., 2017). This type of collaborative and collective engagement shifts the analytic process away from traditional evaluator as arbitrator or white-coated researcher distanced from their inquiry towards inquiry processes that center shared understanding and collective interpretation.

Across the vignettes, data parties also appeared to serve a capacity-building function, although this was more explicit in some contexts than others. Capacity building occurred through modelling analytic dialogue, structuring interpretation with prompts and activities, inviting participants to make meaning from data in real time, and creating opportunities for participants to contribute contextual expertise. In some cases, such as Vignette 1 and Vignette 3, this role was more overtly connected to developing evaluative or analytic confidence; in the other, it remained a more indirect outcome of participation. Future work could examine how data parties might be used not only for collaborative interpretation, but also for sustained community capacity-building beyond a single event.

In addition to enhancing the quality of data interpretation, data parties contribute to a more participatory and equitable evaluation process by shifting the role of interest-holders from passive subjects to active co-analysts. This moves away from extractive or hierarchical models of evaluation, fosters shared ownership of findings, and promotes deeper engagement with the data (Chouinard, 2013; Cousins & Whitmore, 1998). Rather than being passive recipients – or worse, completely divorced from evaluation results – participants brought their expertise, perspectives, and questions to bear on the meaning-making process. This approach cultivates shared ownership and accountability, reinforcing the idea that evaluation is most effective when it is genuinely collaborative. By actively involving participants in meaning-making, data parties help enhance the credibility and contextual relevance of findings and align with the imperative to integrate capacity building (i.e., such as evaluation) and implementation among interest-holders (Hansen et al., 2022). Data parties align with broader calls within the field to include a variety of people and perspectives to ensure that the knowledge produced is grounded in lived experience (Mertens, 2008; Thomas & Campbell, 2020).

Looking ahead, we see considerable potential for data parties to become a more widely recognized method within research and evaluation. These themes indicate that data parties do more than support collaborative practice; they offer a theoretical contribution by showing how evaluative sensemaking can be structured as a shared, relational, and iterative process that challenges hierarchical assumptions embedded in many analytic traditions. When planned and facilitated with care, data parties offer a valuable pathway for producing collaborative, context-responsive, and actionable insights. As we continue to reflect on and refine this approach, we are committed to sharing our experiences and learning from others.

Insights and Advice for Data Parties as a Participatory Method

To conclude, we offer key insights and practical advice for facilitating effective data parties. These insights are drawn from the experiences and observations shared throughout the paper, highlighting strategies to make data engagement more purposeful, inclusive, and impactful. Figure 2 provides recommendations to enhance the value and accessibility of data parties, ensuring they serve as meaningful tools for collaboration and learning. The figure highlights five practical, interconnected practices that strengthen data party implementation. The first, Plan with Purpose, involves structuring data in ways that support meaningful engagement and clear insights. The second, Leveraging Diverse Perspectives, emphasizes including a wide range of participants (including those often excluded from analysis) to enrich understanding and surface alternative explanations grounded in their diverse backgrounds and experiences. Third, Promote Engagement with Interactive Activities through hands-on or technology-supported strategies that encourage active participation and the practical application of insights. Fourth, Make Data Accessible, Relatable, and Interactive by using clear visualizations, plain-language explanations, and adaptable facilitation approaches. Finally, Reflect and Share Learning by inviting feedback on successes, areas for improvement, and future steps.

In the figure, the three interconnected figures symbolize researchers, community members, facilitators, and any other potential participants, illustrating that this process is open to people from diverse backgrounds and experiences. Although the arrows form a cycle to suggest continuous progression, they also depict a dynamic network where each recommendation influences and is influenced by the others.

Figure 2.Recommendations to Strengthen Data-Party Value and Accessibility for Collaborative Learning

This paper has shown how data parties can function as a participatory sensemaking practice across three distinct educational research and evaluation contexts. By presenting these vignettes alongside cross-vignette reflections, we illustrate both the flexibility of the method and the intentional design work required to facilitate it well. Moreover, the lessons learned across these vignettes have relevance for diverse research and evaluation contexts. Although data parties may have emerged as a practical engagement strategy in program evaluation, our findings suggest that they also hold considerable promise for innovative and engaged research. Our contribution is therefore both conceptual and practical: we clarify how data parties support collaborative interpretation while also offering concrete guidance for researchers and evaluators seeking to use them in their own work.

When thoughtfully designed, data parties support joint analysis by enabling interest-holders to interpret data collectively, promote co-production by sharing power and integrating diverse perspectives, and enhance collaboration by building trust and shared ownership of findings. This manuscript illustrates both the benefits and the challenges of this approach, offering practical insights for evaluators and researchers seeking to apply participatory and collaborative principles in their own settings. As participatory analytical practices continue to develop, data parties offer a promising pathway for producing more context-responsive, inclusive, and actionable forms of knowledge.