The relation between methods and recommendations in clinical practice guidelines for hypertension and hyperlipidemia
- OBJECTIVE: To assess the association between methods used to develop clinical practice guidelines and the recommendations that are made.
- STUDY DESIGN: Systematic review of clinical practice guidelines for hypertension or hyperlipidemia.
- OUTCOMES MEASURED: Two people independently appraised guideline methods by using 8 criteria and the aggressiveness of recommendations for treatment thresholds, initial drug selection, and screening.
- RESULTS: We identified 33 guidelines. Only 6 fulfilled 5 or more of the 8 criteria. For 5 of the criteria, fewer than 50% of the guidelines fulfilled those criteria. There was wide variation in recommendations for treatment thresholds, drug selection, and cholesterol screening. Guidelines that did not fulfill the criteria tended to suggest more aggressive recommendations than did guidelines that met the criteria. For 6 of the 8 criteria, guidelines published by specialty societies were less likely to fulfill them compared with guidelines not published by specialty societies.
- CONCLUSIONS: Guideline developers who did not use rigorous methods tended to promote intervening more aggressively for hypertension and hyperlipidemia.
TABLE 2
Variability in fulfillment of methodologic criteria
| Criterion | Fulfilled, n (%) | Not fulfilled, n (%) | No information, n (%) |
|---|---|---|---|
| Main outcomes identified | 10 (30) | 23 (70) | 0 (0) |
| Key stakeholders involved | 21 (64) | 9 (27) | 3 (9) |
| Systematic search and selection | 7 (21) | 26 (79) | 0 (0) |
| Recommendations linked to evidence | 10 (30) | 23 (70) | 0 (0) |
| Benefits and risks considered | 21 (64) | 12 (36) | 0 (0) |
| Resources/costs | 14 (42) | 19 (58) | 0 (0) |
| No industry influence | 23 (70) | 2 (6) | 8 (24) |
| Conflicts of interest stated | 4 (12) | 29 (88) | 0 (0) |
Variability in guideline recommendations*
| BP threshold (mm Hg) to treat hypertension | |||||
|---|---|---|---|---|---|
| Age, clinical scenario | 140 | 150 | 160 | 170 | 180 |
| 50 y, low risk | 4 (25) | 1 (6) | 5 (31) | 4 (25) | 2 (13) |
| 50 y, high risk | 9 (56) | 2 (13) | 3 (19) | 1 (6) | 1 (6) |
| 70 y, high risk | 6 (38) | 1 (6) | 5 (31) | 2 (13) | 2 (13) |
| 70 y, low risk | 8 (50) | 1 (6) | 5 (31) | 1 (6) | 1 (6) |
| Cholesterol threshold (mg/dL) to treat hyperlipidemia | |||||
| ≤230 | ≤270 | ≥310 † | |||
| 50 y, low risk | 3 (19) | 3 (19) | 1 (6) | 9 (56) | |
| 50 y, high risk | 6 (38) | 8 (50) | 0 | 2 (13) | |
| 70 y, high risk | 5 (31) | 3 (19) | 2 (13) | 3 (19) | |
| 70 y, low risk | 7 (44) | 5 (31) | 1 (6) | 3 (19) | |
| *Data are presented as number (%) of patients. | |||||
| † Includes the recommendations “no treatment” and “familial hypercholesterolemia.” | |||||
| BP, blood pressure. | |||||
Associations between recommendations and methodologic criteria
The threshold to treat hypertension did not seem to be associated with fulfillment of methodologic criteria. Differences in recommendations for first-line drugs for hypertension were not strongly associated with any of the criteria. Although not statistically significant, there was a trend for guidelines to recommend all commonly available drugs when methodologic criteria were not met (Table W1, available on the JFP web site: https://www.jfponline.com).
For all but 1 quality criteria (main outcomes identified), fulfilling the criteria tended to be associated with a higher threshold to treat hyperlipidemia. Similarly, guidelines meeting quality criteria tended to give less aggressive recommendations for cholesterol screening than did guidelines not fulfilling the criteria. The criterion on stakeholder involvement was the exception, but this criterion was fulfilled by all but 1 of the guidelines (Table 4).
TABLE 4
Guidelines for hyperlipidemia: relation between adherence to methodologic criteria and recommendations given*
| Guidelines recommending a low treatment-threshold ‡ | |||
|---|---|---|---|
| Criterion | Criterion fulfilled | Criterion not fulfilled | P |
| Main outcomes identified | 3/4 (75) | 7/12 (58) | 1.00† |
| Key stakeholders involved | 5/10 (50) | 3/4 (75) | .58† |
| Systematic search and selection | 0/3 | 10/13 (77) | .036† |
| Recommendations linked to evidence | 1/3 (33) | 9/13 (69) | .52† |
| Benefits and risks considered | 4/10 (40) | 6/6 (100) | .034† |
| Resources/costs | 4/8 (50) | 6/8 (75) | .61† |
| Conflicts of interest stated | 0/2 | 10/14 (71) | .13† |
| Population to screen annually § | |||
| Main outcomes identified | 8 (0–21) | 11 (5–17) | 3 (−8.1 to 14)¶ |
| Key stakeholders involved | 11 (4.8–16) | 1† | −10||,¶ |
| Systematic search and selection | 4 (0–9.8) | 12 (6.2–18) | 8 (−2.3 to 18)¶ |
| Recommendations linked to evidence | 6 (0–13) | 11 (5.3–18) | 5 (−5.5 to 16)¶ |
| Benefits and risks considered | 8 (1.2–15) | 14 (6.0–21) | 6 (−4.1 to 15)¶ |
| Resources/costs | 6 (0.9–12) | 14 (5.8–22) | 8 (−1.2 to 16)¶ |
| Conflicts of interest stated | 0 | 12 (6.6–16) | 12 (−1.8 to 25)¶ |
| *The criterion on industry influence is not included because all the guidelines either fulfilled the criterion or provided insufficient information to assess if the criterion was met | |||
| † P values assessed with the Fisher exact test. | |||
| ‡ Guidelines in which the threshold to treat is less than 310 mg/dL for 3 or more of the clinical scenarios described in the text. Data are presented as proportion (%). | |||
| § Data are presented as percentage (95% confidence interval). | |||
| || One guideline did not fulfill this criterion, so confidence intervals could not be calculated. | |||
| ¶ Difference in percentage (95% confidence interval). | |||
Stakeholder involvement and sponsorship by specialty societies
Guidelines that involved major stakeholders in the development process tended to fulfill the methodologic criteria to a greater extent than did guidelines that did not (Table W2, available on the JFP web site: https://www.jfponline.com). Nine of the 33 guidelines were sponsored by specialty societies. These fulfilled the methodologic criteria less often than did other guidelines (Table 5).
TABLE 5
Relation between specialty society sponsorship and fulfillment of methodologic criteria*
| Guidelines fulfilling criterion | |||
|---|---|---|---|
| Criterion | specialty by specialty society | Sponsored by Not sponsored society | P † |
| Main outcomes identified | 6/9 (67) | 4/24 (17) | .010 |
| Key stakeholders involved‡ | 2/9 (22) | 19/21 (90) | .001 |
| Systematic search and selection | 1/9 (11) | 6/24 (25) | .64 |
| Recommendations linked to evidence | 2/9 (22) | 8/24 (33) | .69 |
| Benefits and risks considered | 3/9 (33) | 18/24 (75) | .044 |
| Resources/costs | 3/9 (33) | 11/24 (46) | .70 |
| No industry influence‡ | 2/4 (50) | 21/21 (100) | .02 |
| Conflicts of interest stated | 2/9 (22) | 2/24 (8) | .30 |
| *Data are presented as proportion (%). | |||
| † Assessed with the Fisher exact test. | |||
| ‡ We did not take into account the guidelines for which we had insufficient information to assess whether the criterion was met. | |||
Discussion
We found that nonadherence to rigorous methods when developing guidelines for hypertension and hyperlipidemia tends to be associated with more aggressive recommendations. We are not aware of other studies that have investigated the relation between methods and recommendations in clinical practice guidelines. The relatively small number of guidelines that met our inclusion criteria limited the power of our analyses, which rarely reached the conventional level of statistical significance (P
Many articles have assessed the methodologic quality of clinical practice guidelines with the use of similar criteria, all these studies found poor adherence to recommendations for guideline development.8-11 Grilli and colleagues found that “the quality of reporting of practice guidelines produced by specialty societies fell short of acceptable methodology” for the 431 guidelines they assessed.10(p104) Shaneyfelt and colleagues found no difference in methodologic rigor between guidelines published by specialty societies and those published by others but decided that methodologic criteria frequently were not met.11 We also found that methodologic criteria frequently were not met, and that they were met less often for guidelines sponsored by specialty societies than for those sponsored by other groups.
