Original Research

Validating the Adult Primary Care Assessment Tool

Author and Disclosure Information




The purpose of the validation was to assess the congruence between the theoretically derived measures and the empiric results in terms of the underlying structure of the principal primary care domains. Although conceptual framework was relied on in the construction of primary care measures, empiric validation was used to reduce the number of items so that the questionnaire became more concise.

The validation of PCAT-AE with the South Carolina sample involved several steps. First, principal component factor analysis was used to explore the structure of the PCAT-AE items and examine its construct validity by determining if the items fell into the hypothesized scales (factors; definitions of measurement-related concepts used in this paper can be found in the Appendix). Factor analysis was also used for item selection and placement into scales based on the pattern of the factor loadings.35 Four criteria were used in deleting items and the determination of the final factors.36-37 Afactor loading greater than 0.35 was considered meaningful and used as a criterion for retaining items. In addition, each retained factor should have at least 3 items with loadings greater than 0.35. All retained items should share the same conceptual meaning or construct. Also, all retained items should not have secondary loadings greater than 0.35.

Second, internal consistency reliability of the primary care scales was assessed by Cronbachs coefficient alpha (a)38 and item-total correlation for items in each domain. Cronbachs coefficient alpha is based on the covariance among individual items in a scale and the number of items. It ranges from 0, indicating total lack of consistency, to 1, indicating complete internal consistency reliability. The item-total correlation is the correlation between an individual item and the sum of the remaining items that constitute the scale. If an item-total correlation is small, the item is not considered to be measuring the same construct that is measured by the other items in the scale. All the retained items exceeded the minimum acceptable item-total correlation of 0.30.38

Third, the Likert scaling assumptions were tested for the final items related to the primary care scales. Likerts method of summated rating scales is based on the assumption that item responses in each scale can be summed without standardization or weighting.39 The underlying assumptions that must be met include: (1) item-convergent validity (tested by item-scale correlations); (2) item-discriminant validity (tested using the scaling success rate, ie, correlation of each item with other items within the same scale is greater than with items from different scales); (3) equal item variance (tested by examining item means and standard deviations and the equivalence of the intraclass correlation and Scotts homogeneity ratio for each scale); (4) equal item-scale correlation (tested by examining the range of item-scale correlations); and (5) score reliability (tested by Cronbachs coefficient a.

Fourth, descriptive statistics were performed for the revised primary care scales, including mean, standard deviation, range, percentile, skewness, kurtosis, and interscale correlation. Since respondents who never saw a specialist did not answer the coordination questions, analyses were performed both with and without those questions, including the coordination domain.



For the HMO group, a total of 350 individuals responded after 3 mailings. Excluding the nonresponses due to wrong addresses and changed plans (n=340), the effective response rate was 53 percent (350/660). The respondents and nonrespondents were not significantly different in age, sex, race, and zip codes of mailing addresses. For the CHC group, a total of 1000 individuals were systematically selected and approached. Among them, 265 refused to be interviewed, 195 were not able to complete the interview prior to their appointment, and 540 completed the interview. Taking only refusal into account, the response rate was 67% (540/540+265). Men were more likely to refuse the interview than women. There were no significant differences in age and race between respondents and nonrespondents. All interviews were conducted by graduate public health students trained in interactive sessions and were completed in 1999.

The sample included 823 adults with an identified usual source of care. Among them, most (69% of HMO and 60% of CHC respondents) indicated a strong affiliation with their usual source of care (ie, all 3 doctors/places were the same). Very few (0.6% of HMO and 1.2% of CHC respondents) indicated the weakest affiliation with their usual source of care (ie, all 3 responses were different). Just over half of respondents (56%) were non-white (primarily black). Over half (55%) had an annual household income under $25,000. Most respondents (76%) had health insurance coverage all year and had been seeing their regular source of care for more than 1 year (82%). Sixty-three percent had seen their regular source of care for more than 2 years. The majority chose their own usual source of care (78%) and did not have trouble paying for their health care (74%). More than half of the respondents made at least 1 visit to a specialist (56%). This relatively high rate may be due to a somewhat elderly sample; more than 20% of the respondents were older than 65 years.

Next Article: