Understanding Practice from the Ground Up

Figure W6 was developed to ascertain self-reported tobacco use status (for all patients) and use of obstetrics and gynecology services and history of hysterectomy (for women). These data were used to determine a patient’s eligibility for tobacco cessation counseling and Papanicolaou tests, respectively. Observation of 30 encounters with each clinician was done to increase the stability for calculating rates for common preventive service recommendations.

A larger issue emerged with the discovery that, after contributing data over the course of weeks or months, members of the practices desired feedback in a timely manner. Practice clinicians and staff were very interested in how they were doing and asked when they would be receiving a report of our results. They did not want to wait 3 years for the completion of the study. Although ongoing analyses were anticipated, these had primarily been designed to ensure completeness of data and to provide feedback to field researchers on areas where clarifications were needed. In response to the emergent desire for feedback, the team generated rapid-turnaround summary reports for each practice. A summary report template was designed to present the descriptive details of the practice, including the practice genogram and a summary of the strengths and weaknesses of the practice’s prevention approach. The final page in the report provided the practice with a series of questions or points for self-reflection that often included process questions, such as “How can this organization become a team?” or “How can this practice deliver preventive services more consistently?” These reports were shared interactively with practices at a debriefing meeting within 2 to 3 months of completing data collection at each practice.

The feedback meetings provided an important opportunity to check the validity of the researchers’ analyses by comparing them with the perspectives of the practice participants. In all the practices, the response was a strong overall validation of the research team’s interpretation of the practice and its structures and processes. During the feedback presentations, the practice physicians and staff consistently made comments like, “Wow, did you ever get us,” or “This is like looking in a mirror.” In a number of sessions, the participants mentioned that the report raised issues about which they were vaguely aware and that the findings were stimulating considerable self-reflection. In several practices, the physicians disclosed that they would be taking actions in the future to modify some of the deficiencies the reports uncovered.

The next modification came as the data were being collected simultaneously in the second and third practices. We realized that despite our efforts to be nondisruptive, participating in the project required extra effort on the part of the practice. Each practice therefore received partial compensation in the form of a $500 certificate for the purchase of books or equipment.

After completing data collection at several practices, it was apparent that patients’ perspectives were still under-represented and that this was limiting the understanding of the practice. To gain further insights into patients’ experiences, beginning with the sixth practice we adopted the patient path approach described by Pommerenke and Dietrich,²⁷ following patients from the time they walk into the practice until the time they leave, using a patient path form for recording activities at different stages of the visit Figure W7. Additional brief open-ended interviews were conducted in the waiting room or examination rooms while patients were waiting.

Although we had asked the research nurses to be more thorough in their descriptions of the community, data from the community and health system were still incomplete. This became even more pronounced when studying practices that were part of health systems. Therefore, beginning with the sixth practice, we included in-depth interviews with individuals from health systems (eg, regional managers and medical directors). A further refinement came with the use of community key informant trees, a systematic process of identifying and interviewing members of the local community surrounding the practice.^24,28 These interviews of patients, church leaders, and other individuals from the community began with the ninth practice.

Once all the modifications were incorporated, the final case study design provided data at each of the 6 levels as shown in the Table 1. Particularly detailed data were available at the clinician, encounter, and practice levels. For example, at the clinician level the data included perceptions of roles as ascertained through the in-depth interviews, as well as actual behaviors recorded in the encounter field notes and chart audits. Insights on the structures and process of the practice were obtained through unstructured observations of the practice, structured checklists, written documents, and interviews. Supporting data were collected on patients’ perspectives, the community, and the health system that provided contexts for the practice case studies.