A Group-Level Assessment, or GLA, is a participatory research method that provides multiple opportunities for participants to give input and collaboratively create actionable change (Vaughn et al., 2011). GLA participants share their lived experiences and gain empowerment to work together for change benefitting not only themselves but also those within their community (Lindquist-Grantz & Vaughn, 2016). When designing my research study, I recognized the need for a research method focusing on equity and transformation due to the stigmatization and marginalization that participants often experience (Garriott, 2020). This research explored the college experiences of first-year undergraduates (n=14) placed into remedial courses (rather than traditional college-level coursework) due to their test scores (Dorhout, 2021). Initially, the students had an unfavorable view of their placement into remediation and its resulting stigmatization, but instead of withdrawing from college they chose to come to the metaphorical research table to share their experiences and be viewed as a valuable community expert rather than a stigmatized member of the community (Guy & Boards, 2019).
However, when the COVID-19 pandemic suddenly closed campuses and the university quickly pivoted to online learning, my planned in-person GLA became out of reach. Rather than temporarily shelve the study and wait for campus activities to resume, I modified the original GLA research plan. As a result, the modified plan provided robust data, rich analysis, and a clear understanding of the contextual aspects the participants experienced in the same way that a traditional GLA does, thereby demonstrating that online participatory research is possible even if it’s a Plan B.
This research brief describes the online adaption of a traditional GLA (Vaughn, 2014; Vaughn & Lohmueller, 1998); to stay true to the methodology of a GLA, I refer to the online adaptation as a modified GLA (mGLA). First, I briefly overview GLA, then describe the mGLA and the necessary modifications for it to be completed online. As I share the dilemmas encountered during the mGLA, I explain how participants chose to navigate the experience, providing other researchers with a model highlighting the flexibility required in participatory research. Finally, I reflect on my own learning as the primary researcher attempting to modify an exemplary participatory research tool into an online research tool.
Overview of Traditional GLA
Researchers consider a GLA as a concrete, time-saving methodology for use with large groups (Arthur & Guy, 2020; Lindquist-Grantz & Vaughn, 2016). The seven distinct phases of a GLA situate the participants into multiple aspects of the research process, including data generation, analysis, and prioritization; the inclusion affirms the participant’s personal value to both the research and researcher (Vaughn et al., 2022).
The seven phases of a traditional GLA are presented below in Table 1. The first phase, Climate Setting, begins by establishing and affirming the participants’ individual value, gaining consent, introducing participants, building trust, and describing the process. Once the climate is established, all participants have the opportunity to share their thoughts by answering open-ended prompts displayed throughout the room on poster paper during Phase 2, Data Generation. Typically, there is one prompt on each poster, and often the posters are colorful with simple designs or drawings on them. Prompts can be either broad or specific, but they should attempt to balance ascertaining the strengths, weaknesses, positives, and negatives of the topic. Additionally, Vaughn and Lohmueller (2014) suggest the use of both serious and fun prompts, such as a song or movie to represent the participants’ thoughts.
After sufficient time to answer all prompts, the participants enter Phase 3, Appreciating Responses, and review the answers provided to each prompt; in Phase 4, Reflecting Individually, participants reflect on and distinguish areas of strength, weakness, and potential growth within their community. Next, group members verbally share their insights in small groups in Phase 5, Understanding Together. Phase 6, Selecting Priorities, begins with a large group conversation that encourages all small groups to share thoughts, listen to others, and prioritize main findings. In the final phase, Action Items, the entire group decides on what they value within their community and create actionable plans to improve their community.
Because of its interactive nature, GLAs provide participants with opportunities to connect with members of their community and build relationships. However, during the COVID-19 pandemic, this relational, community-building aspect seemed impossible.
mGLA
Additional aspects of traditional GLAs also presented difficulties in completing the study online. First, GLAs typically occur over several hours (Vaughn, 2014; Vaughn & Lohmueller, 2014). Following the traditional three-hour time allotted for a GLA could cause screen fatigue and potentially impact the results. The second factor also relates to timing: the participants had previously expressed concerns that their class and work schedules did not provide time for the multiple, back-to-back free hours needed for the GLA. Rather than delay the study and this opportunity for students, I adapted the GLA to fit the participants’ circumstances by abbreviating the meeting times.
Vaughn and Lohmueller (1998) describe an abbreviated GLA and advise the longer session, but to establish trust and truly share power with the participants, this study was condensed and the phases divided into pre- and post-session activities. The modifications remain true to the community-based, participant-centric spirit and purpose of the GLA method, but the differences are significant. As such, I classify the modified GLA as an mGLA and the combined, altered, or condensed phases of the mGLA are differentiated with an “m” before them. Table 2 delineates the phases within the mGLA and compares them to the traditional GLA.
The mGLA replicated foundational features of a GLA—like personal relevance, collaborative analysis, and shared decision-making—and adjusted them to innovatively conduct participatory research during the COVID pandemic. The mGLA shifted pieces of the original seven GLA phases into pre-, online-, and post-session work (Table 2).
Pre-session Modifications
Pre-session work for the mGLA began five days prior to the online session. The m1 phase, Generating Data, included participants receiving an email containing the following five items: an overview of the study, a shared Google slideshow with the mGLA’s open-ended prompts, a Zoom link for the online session, the consent form, and my personal contact information for any clarifications or technological difficulties. The overview asked participants to respond to the 15 open-ended prompts on the shared Google Slides document within the next 72 hours; a rationale for the pre-meeting data collection is discussed later in this piece. Participants placed their answers in the numbered table and were to remain consistent on every slide in the number selection (See Figure 1). The prompts on the shared slideshow were introduced with a welcome slide followed by a slide with directions for completing the mGLA prompts. Slides 3–17 contained an open-ended prompt and a numbered table (see Figure 1) and slide 18 expressed my thanks to all the participants for their time. An important aspect of the directions on slide 2 included responding to uncomfortable answers. Though the mGLA answers were anonymous, participants had the option to skip any prompt needing clarification or causing them discomfort while considering potential answers.
In m2, Appreciating Responses, participants spent time appreciatively reviewing the responses of others prior to our online meeting. Two days before the online meeting, participants received an email asking them to review the prompts and consider the answers on the shared slides, look for similarities, intriguing answers, areas of strength, and potential areas of growth. Even if a participant did not answer a specific prompt, they should still read and consider the responses of their peers. In addition to reading and appreciatively considering the responses of others, participants were asked to leave the responses as-is, which included leaving open spots or misspelled answers on the tables. Together, m1 and m2 set the stage for small- and large-group conversations during the online session.
Online Session Modifications
I welcomed each participant to the study as they joined the Zoom session individually. Three participants did not enable their cameras, and when I broadly addressed blank screens and explained the video feature, they either could not or did not enable it. No one asked questions regarding the video, but instead, all asserted they were ready to begin the study. Once all students indicated comfort with the technology, I began reading the protocol script to gain consent. Shanda Scott et al. (2015) argue that students placed in developmental education often have limited technological connectivity or proficiency. To mitigate any potential technological issues, I asked the participants to verbally consent to participate in the study as we continued through m3 Setting Climate. The protocol script contributed to the session’s climate by including multiple statements regarding the students’ value and the participatory nature of the mGLA such as: “You are the expert,” “I need to learn from you,” and “Only you can answer these questions accurately.” Students verbally consented and granted permission to record the session.
Next, participants briefly introduced themselves. If their Zoom screen names differed from their preferred names, I renamed the participant and verified my accuracy. I overviewed the study’s plan, explained the Zoom features we might use, and asked what needed to be clarified. One participant mentioned that they had been unable to access the slides due to unreliable internet access. Another participant did not access the document because they were unfamiliar with Google Docs, so they did not attempt it. Both participants understood the potential of being disqualified from the study due to technological barriers, even if the barrier was self-imposed. With this added information, I made a split-second decision to embrace equity and include these participants who wanted to come to the research table; however, this equitable decision caused further alterations to the modifications I had already designed for the online session.
The altered m4, Reflecting Individually, included time for participants to reflect and identify themes within all participants’ answers at the beginning of the session. Based on the inclusion of the two participants who had not previously answered the prompts, the other participants suggested and unanimously decided to add another modification to the individual reflective time of m4. Participants suggested I share my screen with the first prompt visible, read the prompt aloud, and add the verbal answers from the participants who previously had access issues. In addition to their modifications, I suggested all participants consider or appreciate the responses as I read them and immediately member-check accuracy in recording their thoughts. After each participant agreed to these modifications, we proceeded through the study and provided all with equitable opportunities for input. Though not within the original mGLA plan, these participant-driven modifications were based on equity and empowerment and contributed to community building.
Before moving to the next prompt as planned, one participant boldly shared their thoughts about the themes emerging from the first prompt. The verbalization was not part of m4, but was organic and welcomed. Other participants recognized this empowerment, followed suit, and continued the discussion by interjecting their thoughts regarding the groups’ answers. Cognizant of 14 additional prompts requiring answers before we could discuss the themes in small groups, I inserted myself into the conversation during a lull and reiterated the participants’ desire to have a time cap. In doing so, I also explained that the participants were community experts, and if my plan needed further adaptations we should discuss and decide now. To my delight, they embraced this empowerment, took the lead, and decided to reflect and discuss the responses to each individual prompt rather than discussing multiple prompts later. They asserted that discussing one prompt at a time would streamline the process for them. We further adjusted and combined m4, Reflecting Individually, and m5, Understanding Together (Table 3).
As the facilitator, my responsibilities now included the following: read the prompt, collect outstanding answers on the prompt’s table, and restate all answers for the prompt while participants reflect on the answers. As the large group discussed their findings within each prompt, I took notes in the comment bar at the bottom of the Google Slideshow. Before we moved to the next prompt, I requested that participants review and member-check my notetaking accuracy. Once participants agreed on the notes, we moved to the next prompt. The group discussion on each prompt lasted between three and five minutes, including member-checking.
In following the lead of the participants through m4 and m5, we sacrificed small-group discussions before a large-group discussion based on the flexible and adaptive nature of GLAs (Vaughn & Lohmueller, 2014). We also sacrificed quiet reflection time since I verbally reviewed all answers before the discussion. While not ideal, we made this concession based on time constraints and participant preference. The omission of quiet reflective time felt more comfortable to the participants, and one expressed, “It is good to finally talk to somebody at the school, like hear their voice instead of always reading emails and class notes. None of my classes meet live, so I haven’t talked to anyone else except this group.”
After the participants discussed each prompt as a large group, we shifted the discussion to Phase 6, Priorities and Actions. Throughout the group discussion, the participants forwarded ideas regarding the groups’ strengths as both students and community members. The group agreed on a strength they each exhibited, but also identified how that strength could turn into a weakness if they relied on it too much. Likewise, they focused on their perceived weaknesses and barriers, but as they talked through these, they recognized potential paths for overcoming them. Finally, the participants addressed potential areas of growth both for themselves and for the institution. During the remaining few minutes of the mGLA online session, the group chose three actionable next steps based on this final group discussion in Phase 6, Priorities and Actions.
The mGLA plans experienced slight modifications throughout the live session; still, we included the essential elements of a GLA. Specifically, the participant-adjusted mGLA offered an appreciative-relational experience, included data generation, and provided reflection on data and group discussion, as well as a collection of actionable items. The original mGLA design’s goal included a live session time limit of approximately 60 minutes based on early feedback from participants. With the additional modifications, the total online session time was 68 minutes long.
Post-session Modifications
A traditional GLA is complete once participants finalize action items; for the mGLA, I added an additional step: m7, Follow-up. Because our online session was time sensitive, I wanted to provide the participants with an opportunity to further member-check the transcripts and the Google Slides. Therefore, each participant received an email thanking them for their participation, flexibility, and transparency in the study. They also received a copy of the final transcripts and the slides to review, and although the participants had already received a copy of the consent form, I included it again to offer easy access to all documents (Scott et al., 2015).
Reflection and Considerations
As the participants and I collaborated in the mGLA, we experienced transformation. We gained knowledge about one another and agreed on actions to promote a more socially-just experience for students who often experience marginalization. The modified method, however, included hiccups involving technology and online etiquette (netiquette), time constraints, and relational barriers that researchers designing online, participatory studies should consider.
Technology and Netiquette
Though undergraduate students are often assumed to be digital natives capable of utilizing various technological applications with ease, the participants’ familiarity with technology did not guarantee a level of proficiency with unfamiliar features or the use of netiquette in an online study (Kopp et al., 2019). One participant commented that until our session, they had never used Zoom and debated joining the session because of the need to learn “something else on the computer.” Another participant displayed only the top of their head during the online session, a participant joined the group while lying down in their bed, and another walked outside during the session thereby continuously jostling their screen during the meeting. Interestingly, the other participants made no reference to the visual hiccups of the session, though they felt distracting to me.
During the online session, there were moments when the participants and I were unable to hear one another due to background noises such as phones ringing, animals barking or chirping, and others loudly entering the room or home. Some participants appeared bothered by the noise as evidenced in their facial expressions, while others appeared undisturbed. I acknowledged the noise with statements such as, “What type of bird do you have?” By acknowledging and relating to the noise in the participant’s background, I contributed to the community-building focus of the mGLA and avoided alienating any participant. Though I had the capability to mute students with excessively noisy backgrounds, this could interfere with the open-communication, participatory nature of a GLA. Still, at one point I chose to mute someone who answered their phone during our session. Once the participant put their phone away, I unmuted them without addressing the action; from that moment on, they fully participated as if there had been no disruption. Future considerations for researchers utilizing the mGLAs should include more upfront training on what to expect during the session, how to fully participate, and general information on netiquette both during the pre-session activities and during m3, Setting Climate. Similarly, future researchers should explore other online tools, for tools such as Padlet or Teams may also provide rich data.
Time Constraints
Even with planned pre- and post-session activities, we found it challenging to navigate a succinct mGLA. Yet, based on conversations and feedback from the participants before the session, I knew the participants considered a timely event of paramount importance. Before the study, one participant shared, “I have 3 tests this week, but I can talk to the group for an hour. Only one hour, right?” Others communicated work or family obligations before or after our session. As such, I assert that the decision to collect pre-session data, though untraditional, benefitted the participants. While we moved through the study, I closely monitored time and announced our progress every few slides. One participant later commented that the quick pace and progress reports eased the pressure they felt regarding the time they were giving to the study. Another participant commented that they “did not feel rushed” and believed that we structured the session in a way that respected their time.
The succinct online session demonstrated that the participants and their values were respected. Though it was challenging to keep the session short, the decision aligned with key principles of participatory action research, such as placing the needs of the participants over the demands of the research, respecting participants, and involving participants in multiple aspects of the research, including its design. The participants set the time limits, and my role was to adhere to them.
However, I found it difficult to be present in the moment while being a rigid timekeeper. So, as we approached the 60-minute mark, I needed to dispense with the original, planned ending that rehearsed enjoyable aspects of their education. With this planned closure, I hoped that participants would log off feeling encouraged and desiring more relationships within this community. While I was initially disappointed to forego the closure, the transcripts later revealed that this data emerged organically, as often happens in traditional GLAs (Dorhout, 2021; Vaughn & Lohmueller, 1998).
Relational Barriers
GLAs often recruit participants via snowball sampling (Vaughn & Lohmueller, 2014); however, COVID-19 restrictions limited peer interactions and this recruiting strategy. Instead, participants enrolled in the study by completing a Google Form posted on the learning management system by their faculty members. In the responses I received about the posting, participants expressed eagerness to join the study. Yet, almost one-third of those who originally agreed to participate did not. Interestingly, I received no further communication from the absent participants, even though they had previously communicated multiple times with questions regarding the study. They may have lost their zeal to follow through because of uncertainties about the research process, their lack of connection to and trust of others, or a perceived lack of self-importance in this study.
Due to both COVID-19 restrictions and the nature of participants’ asynchronous classes, only one of the participants had met another university student, staff, or faculty member before the study began at the midway point of a semester. As such, forming the foundation for a relational, participatory community in 60 minutes presented a lofty goal. Jordan and Schwartz (2018) describe being transparent and vulnerable as difficult if there is no underlying relationship, and yet GLAs depend on the participants’ willingness to be transparent and participate in discussing hidden issues or power imbalances (Vaughn, 2014). Three participants affirmed my concerns regarding trust and community-building as they refused to enable their video screen at any point during the session. Though vocal and involved throughout the online session, their name-only screens represented a relational barrier both among the participants and between these participants and me. One of the participants with a name-only screen later shared in an email, “I didn’t want the group to see me in my [work] uniform.” Another explained their blank screen as, “I didn’t mind talking to the people, but I didn’t know them and didn’t want them to see me at home.” They were willing to participate and share their thoughts, but they were not willing to truly connect or trust this new community.
Despite the less-than-optimal conditions for the mGLA, this study models researcher flexibility and meeting research participants at the level that makes sense for their context. The relational foundations laid through the pre-session correspondence and online session format eventually contributed to the participants’ openness. When participants sent pre-session emails or called me, I replied within 24 hours and always included, “What other questions or challenges do you currently have?” The open-ended question provided an opportunity for participants to ask questions regarding tutoring, counseling, support groups, and a food pantry. I answered each question and provided referrals. Beyond these relational elements, the consent form acknowledging the participants as experts and the participant-driven format of the online session strengthened our burgeoning relationship and empowered the participants to share transparently. However, they shared on their own terms, which sometimes included a name-only screen.
Despite the relational, timing, and technological barriers we experienced during the online session, the participants displayed positivity as they signed off. Participants appreciated “finally seeing someone” to which another quickly added, “Yeah, there are people, not robots, at the school.” Beyond these spoken sentiments, those who had their screens on smiled and waved goodbye to one another, as if they had missed daily social interactions with their peers and desired future meetings. These subtle visual and verbal cues indicated a relational online experience and a budding community. The mGLA made strides in accomplishing its relational goal, and its participatory nature shone as it drew others in and empowered them to bring about change.
Conclusion
The mGLA remained true to the spirit of a traditional GLA in that it focused on the participants and their needs, provided opportunities for the participants to openly share their thoughts in various forms, and made room for them to decide together on priorities and next steps. Though the online adaptation was messy at times, this Plan B worked to engage the participants and generate data, but it also laid the foundation for a group of students committed to transforming their community both for themselves and for others.
Conflicts of interest
I have no known conflict of interest to disclose.