Methodological Progress Note: Group Level Assessment
© 2019 Society of Hospital Medicine
Prompt Development
Similar to the development of questions for interview and focus group guides, creating effective prompts is a critical component of data collection in GLA. Prompts are statements worded as incomplete or fill-in-the-blank sentences that should be open ended to allow participants to respond with their own thoughts and experiences. Prompts that resemble the beginning of a sentence (eg, “The biggest challenge we face is…”) encourage honest reflection rather than questions that can make participants feel like they are being evaluated. We recommend varying the number of prompts based on the group size: approximately one chart and prompt per person attending, with a maximum of 35 prompts at one session.1 This allows for sufficient variability in the responses generated without being overwhelming or too time-consuming. For example, Choe et al. developed a pool of 51 unique prompts addressing their research questions and then used 15-32 prompts in each GLA session, depending on the number of participants. 8 Prompts should be written with some purposeful redundancy, targeting the research question from several angles. The emphasis should be on the content’s alignment with the research questions rather than the actual wording of the prompts as a way of ensuring that the generated data is both valid and useful.
Prompts should also vary in format, style (eg, different color markers, pictures, fonts, etc.), and placement on each flip chart page. An individual flip chart can include multiple related prompts: for example, “split-halves” in two columns or rows (ie, the best part/worst part). Taken as a whole, the flip charts and accompanying prompts create different lenses for gathering participant perspectives on the research questions. See Appendix Table for suggested prompt characteristics and examples from a hypothetical study related to pediatric healthcare.
GLA prompt development will ideally occur in collaboration with an advisory team comprised of representative members from each of the stakeholder groups. Using a participatory research approach in the research design and preparation phases ensures that GLA prompts are understandable and relevant to participants and are able to appropriately capture the underlying purpose of the study.
Description of the Seven Steps in GLA
In step one, climate setting, the facilitator provides an overview of the session, including a description of the GLA rationale and process. Typically, an icebreaker or brief introduction activity is conducted. Step two, generating, is a hallmark step of GLA in which participants walk around and respond to prompts prewritten on flip charts hung on walls in a large room. Participants use markers and respond to each prompt by either providing a unique comment and/or corroborating an existing comment by adding a checkmark or star. During this step, organizers typically play music and encourage participants to enjoy food, chat with fellow participants, and leisurely move from prompt to prompt in any order. Step three, appreciating, is a brief interim step where participants take a “gallery walk” and view responses written on the charts.
In step four, reflecting, participants reflect on the data and briefly write down their thoughts about the responses generated in the session. In step five, understanding, smaller groups synthesize responses across a subset of charts and report their findings to the larger group. Depending on the size and composition of the larger group, small groups of four to seven people are formed or assigned. Each small group is assigned a subset of approximately four to six charts. Using thematic analysis, participants look for relationships among the responses on their assigned charts, referring to individual responses as evidence for the main findings. Groups will take notes on the charts, circle key phrases, or draw arrows to show relationships in the data and thereafter develop themes. As each small group reports their findings, the facilitator will keep a running list of generated themes, ideally in the participants’ own words. Step six, selecting, involves participants discussing, further synthesizing, and prioritizing data. Step six can occur as a facilitated large group discussion or in a form in which participants can remain in the same small groups from step five and work together to complete this further step. Themes across all of the small groups are consolidated and developed into overarching themes. Step seven, action, includes planning the next steps to address priorities.
