We report on the validation of the Consumer/Client Primary Care Assessment Tool Adult Edition (PCAT-AE). Its companion instrument for children (PCAT-CE) was previously validated.30 Specifically, we assessed the congruence between the theoretically derived measures and the empiric results in terms of the underlying structure of the principal primary care domains within a diverse sample of populations including health maintenance organization (HMO) members and community health center (CHC) users. The validation process also served to reduce the number of items needed to assess the adequacy of primary care.
The study participants were members of 2 health plans in 2 counties of South Carolina. Both counties are part of Columbia, the states capital and third largest city. One of the health plans (referred to as HMO) is licensed as an independent practice association (IPA) HMO model, in which primary care physicians act as gatekeepers and health care managers. Referral to specialists must be made through primary care physicians, and specialists must be affiliated with the HMO. The primary market has been large group employers, including employees of the state agencies and national and regional companies. Members of this plan are primarily from middle-income households. The other health plan (referred to as CHC) is a coalition of 12 Columbia-based health and social services provider organizations, including the county hospital, health department, department of social services, community health centers, and other social service agencies that provide services to lower income persons, such as Medicaid recipients and low-income households. These 2 plans were selected because they represent typical South Carolina managed care organizations and health plans for low-income individuals, respectively. Samples drawn from these 2 plans allowed us to test the reliability of PCATwith a diverse sample of populations, including both middle-income and low-income individuals using regular physician offices and community health centers, respectively.
Estimation of the sample size for this study involved several steps. First, an estimate of the likely proportions or means and standard deviations for each primary care measure was derived from a previous study.25 When data were not available, a conservative estimate (eg, a larger standard deviation or proportion closer to 50/50) was made. Second, the estimates of the proportions, means, and standard deviations for the dependent variables were entered into the standard sample size formula for a two-group, cross-sectional sample. Using a 95% confidence interval, the largest sample size required was 300 per group. The CHC group was oversampled because of additional planned within-group analyses (not the focus of this paper). Finally, the desired sample size was adjusted for anticipated survey nonresponse (anticipated to be higher for a mail survey than a face-to-face interview).
For the HMO group, a mail survey was used since it was deemed most efficient. In 2 previous longitudinal studies of the same HMO, we used mail survey and telephone interviews alternately with a cohort of HMO members and obtained comparable results.31,32 For this study, we sent a letter with a PCAT-AE questionnaire to 1000 randomly selected adult members to invite them to participate in the project. Because of known frequent changes in addresses, we recruited the non-HMO plan individuals and conducted in-person interviews at all the community health center sites where members came to the clinics for non-urgent visits. Patients were systematically approached while waiting for their scheduled appointment (ie, every nth patient based on expected visits for a particular site) and recruited for the study during a period of 4 weeks for each site.
Identification of Primary Care Source.Three questions were developed to identify an individuals usual source of care and the strength of that affiliation: (1) Is there a doctor or place that you usually go if you are sick or need advice about your health? (usual source), (2) Is there a doctor or place that knows you best as a person? (knows best), and (3) Is there a doctor or place that is most responsible for your health care? (most responsible). Aperson was considered to have a usual source of care if he or she answered positively to any 1 of the 3 questions (95% for the HMO plan and 90% for the low-income plan). Anegative answer to all 3 questions rendered the individual as not having a usual source of care.
An algorithm based on response to these 3 questions identified the strength of affiliation with the primary care source. If all 3 physicians/places were the same, this was considered evidence of a strong affiliation. If the response to the usual source question was the same as for either of the other 2 questions then that site was used although the affiliation was considered less strong. If the response for a usual source question was different from the other 2 responses but the other 2 responses were the same, then the site where both were the same was used (weak affiliation). If all 3 responses were different (weakest affiliation), then the site identified for usual source was used. All subsequent questions asked about this specific person or place. For those with no identifiable source of primary care, subsequent questions were asked about the last place that was visited.