Conducting The Direct Observation of Primary Care Study Insights from the Process of Conducting Multimethod Transdisciplinary Research in Community Practice
Research Design
The research team began refining study questions and developing methods. A critical event occurred during a discussion of methods for measuring the content of outpatient family practice visits. Jason Chao, a family practice academician, enumerated these methods:“…chart review, patient questionnaire, billing data. One could do direct observation, but you can’t do that.” As everyone nodded agreement, his colleague Robert Kelly interrupted, “Why not? Why can’t you do direct observation?” The group listed many good reasons: intrusiveness, unacceptability to patients and clinicians, expense, and the potential to bias behavior. However, the question “Why not?” remained and created a shared sense that direct observation of real world family practices represented an opportunity to make a unique contribution. The group decided to include direct observation as a major measurement technique and to add a methodologic goal of establishing the validity and reliability of nonobservational techniques for assessing the content of outpatient medical practice. An additional advance occurred with the publication of the Davis Observation Code (DOC)46 that classified patient visits into 20 different behavioral codes measured in 15-second intervals. Lead author Edward Callahan agreed to become a collaborator.
Limited existing research on the content of community primary care practice meant that the group would have difficulty in anticipating all content areas worth measuring and questions worth asking before immersing themselves in community family practice settings. Therefore, Drs Crabtree and Miller were asked to join the team to design a multimethod approach that integrated quantitative and qualitative methods.47 Project design was pursued further in research team meetings, telephone conversations, and interactions with out-of-town collaborators during national professional meetings. These face-to-face meetings were essential to developing the trust, communication, and shared vision necessary for a transdisciplinary multimethod study.
Conversations with local family physicians soon revealed that preventive service delivery, although an important aspect of family practice, was not a sufficiently compelling research question to engage a new practice-based research network. A broader focus, such as the content of family practice, would engage the largest number of clinicians and be less likely to bias clinician behavior during direct observation. At the suggestion of practicing family physician Michael Rabovsky, in whose office the protocol was being pilot tested, the study was expanded to address the Medicare Resource-Based Relative Value System (RBRVS)-based billing system. Health economist Daniel Dunn, who helped develop the RBRVS,48 was invited to participate.
Based on discussions with practicing family physicians, a strategy was developed for recruiting practice-based research network members. Members of the Ohio Academy of Family Physicians (OAFP) in Northeast Ohio were targeted to facilitate easy meeting of practices and travel of study teams to practice sites.45 A letter describing the study and proposed network was sent to all 531 OAFP active members in the area. A total of 138 physicians responded and formed the fledgling Research Association of Practicing Physicians (RAPP) network. A working relationship was established with the NorthEast Ohio Network (NEON), a practice-based research network of 6 community residency training sites affiliated with the NorthEast Ohio Universities Colleges of Medicine, directed by William Gillanders (and later Valerie Gilchrist). NEON physicians were trained in National Ambulatory Medical Care Survey (NAMCS)49 data collection techniques and provided the opportunity to evaluate the validity of the NAMCS methods compared with direct observation. These development activities were supported by considerable in-kind contributions of investigator time from the participants’ institutions.
Pursuit of Funding
A research concept paper was sent to the AHCPR for feedback. The response indicated that intervention studies were more compatible with funding priorities than the proposed observational study. The critique also pointed out “fatal flaws” engendered by direct observation methods and expressed skepticism that community physicians would allow such observation of their practices. These concerns were addressed with pilot data and a strengthened argument about the need for efforts to understand practice before attempting to change it. An investigator-initiated (R01) grant application was submitted to the AHCPR. A secondary assignment to the National Cancer Institute (NCI) was requested because of the clinical preventive service delivery focus and the important potential of understanding family practice and competing demands for the subsequent design of interventions to enhance cancer prevention and control.
The initial application was favorably reviewed and received a priority score near the funding line. In response to advice from NCI and AHCPR program officers, the research team allowed the application to be considered for funding during 3 upcoming NCI council meetings. Regular letters to research network members kept them informed of the funding status. After 1 year of narrowly missing the funding line, the grant application was revised and resubmitted in response to the scientific review committee’s critique, with increased emphasis on the implications of the study for cancer prevention and control. It was funded by the NCI, with an additional grant from the Robert Wood Johnson Foundation Generalist Physician Faculty Scholars Program, to develop communication and clinician-initiated research in RAPP and additional methodologic and descriptive aims.
