STUDY DESIGN: Two family physicians attempted to answer 20 questions with each of the databases evaluated. The adequacy of the answers was determined by the 2 physician searchers, and an arbitration panel of 3 family physicians was used if there was disagreement.
DATA SOURCE: We identified 38 databases through nominations from national groups of family physicians, medical informaticians, and medical librarians; 14 of these databases met predetermined eligibility criteria.
OUTCOME MEASURED: The primary outcome was the proportion of questions adequately answered by each database and by combinations of databases. We also measured mean and median times to obtain adequate answers for individual databases.
RESULTS: The agreement between family physician searchers regarding the adequacy of answers was excellent (k=0.94). Five individual databases (STAT! Ref, MDConsult, DynaMed, MAXX, and MDChoice.com) answered at least half of the clinical questions. Some combinations of databases answered 75% or more. The average time to obtain an adequate answer ranged from 2.4 to 6.5 minutes.
CONCLUSIONS: Several current electronic medical databases could answer most of a group of 20 clinical questions derived from family physicians during office practice. However, point-of-care searching is not yet fast enough to address most clinical questions identified during routine clinical practice.
Family physicians and general internists report an average of 6 questions for each half-day of office practice,1-3 and 70% of these questions remain unanswered. The 2 factors that significantly predict whether a physician will attempt to answer a clinical question are the physician’s belief that a definitive answer exists and the urgency of the patient’s problem.4
Gorman and colleagues3 reported that medical librarians found clear answers for 46% of 60 randomly selected questions from family physicians; 51% would affect practice. The medical librarians searched for an average of 43 minutes per question. In a second study,5 medical librarians used MEDLINE and textbooks to answer 86 questions from family physicians. The MEDLINE searches took a mean of 27 minutes, and textbook searches took a mean of 6 minutes. Search results answered 54% of the clinical questions completely or nearly completely. Physicians estimated that the answers would have a “major” or “fairly major” impact on practice for 35% of their questions. MEDLINE searches provided answers to 43% of the questions, while textbook searches provided answers for an additional 11%.
Many physicians do not have the searching skills or access to the range of knowledge resources that librarians use. Even if they did, they do not take the time to conduct such searches during patient care. One study1 found that physicians spent less than 2 minutes on average seeking an answer to a question. Thus, most clinical questions remain unanswered.
Electronic medical databases that provide answers directly (not just reference citations) may make it easier for clinicians to obtain answers at the point of care. We found no systematic evaluation of the capacity of such databases to answer clinical questions. We conducted this study to determine the extent to which current electronic medical databases can answer family physicians’ point-of-care clinical questions.
We solicited nominations for potentially suitable databases from multiple E-mail lists (including communities of family physicians [Family-L], medical informaticians [FAM-MED] and medical librarians [MEDLIB-L, MCMLA-L]) and through Web searches. A selection team consisting of 3 family physicians (J.S., D.W., B.E.) and a medical librarian (none of whom had financial relationships with any databases) determined whether the nominated databases met our inclusion criteria Table 1.
More than 1200 clinical questions had been previously collected from observations of family physicians during office practice.1,5 These questions had been classified by typology (eg, Is test X indicated in situation Y?) and by topic (eg, dermatology).1 We selected questions from these sources that were categorized among the most common typologies (8 of 68 typologies covering 50% of the questions) and the most common topics (7 of 62 topics covering 43% of the questions). These combinations of typologies and topics accounted for 272 (23%) of the 1204 questions.
If necessary, each question was translated by 2 physicians (B.A. and D.W. working together) to meet the following criteria: (1) clear enough to imagine an applicable clinical scenario, (2) answerable (ie, the question could theoretically be answered using clinical references without further patient data regardless of whether an answer was known to exist), (3) clinically relevant, and (4) true to the original question (ie, containing the information need and the modifying factors of the original question).
Each question was then independently proofread by at least 2 other physicians and translated again if necessary. Thirteen questions (5%) that did not meet these criteria after a second translation were dropped. Forty-seven questions (17%) that referred to information needs that could be adequately answered using the Physicians’ Desk Reference6 were dropped (eg, Are Paxil tablets scored?). The remaining 212 questions represented 8 typologies.1 Two or 3 questions were randomly selected from each typology for a total of 20 questions Table 2.