Vertebroplasty, cognitive dissonance, and evidence-based medicine: What do we do when the ‘evidence’ says we are wrong?

TWO NEW STUDIES UPSET ESTABLISHED BELIEF

This is why I am having such a hard time digesting the results of the trials by Buchbinder et al² and Kallmes et al,¹ published in the August 9, 2009, issue of the New England Journal of Medicine. Both were randomized controlled trials that used sham surgery rather than conventional medical treatment as the control. The sham procedure in each trial was the same as the intervention, with local anesthetic infiltration of the periosteum and mixing of the cement (so that the patients smelled its distinctive odor), but without placing the needle into the vertebra and injecting the cement.

In the study by Buchbinder et al,² the real treatment had no benefit in any primary or secondary end point. This study did not allow crossovers.

In the study by Kallmes et al,¹ more patients who received the real treatment reported clinically meaningful improvement in pain (a secondary end point), but the difference was not quite statistically significant (64% vs 48%, P = .06). In this trial, patients were allowed to cross over to the other study group after 1 month, and significantly more patients crossed over from the sham surgery group to the active treatment group than the other way around (43% vs 12%, P < .001).

My first instinct was to pick through the papers for flaws that would invalidate the results— and there were some problems. Both studies were initially planned to include more patients and therefore to have greater statistical power, but they were reassessed because of slow enrollment. In the study by Kallmes et al,¹ the difference in clinically meaningful improvement might have reached statistical significance if the trial had been larger. The study by Buchbinder et al² was a multicenter trial, but one center accounted for 53 (69%) of the 78 patients. Could this have biased the results?

The surgeon in me also seized for a while on the idea that since all of the interventions in both studies were done by interventional radiologists, the problem may have been in patient selection and that radiologists are not as astute as we are. However, even a surgeon’s ego cannot support this interpretation.

As I looked in more detail at the response I had written to these trials, I realized these criticisms were hardly fatal flaws, and the fact that two separate well-designed studies reached the same conclusion enhances their validity.

One concern that does bear some scrutiny is that the trials were too small to identify subgroups that may benefit from the procedure. In my experience, vertebral augmentation seems to have better results with certain types of fractures. Patients with a mobile pseudarthrotic cleft pattern of fracture seem to do much better than those with the more common nonmobile fracture.

THE POWERFUL PLACEBO EFFECT

Many commentaries on these two trials have discussed a famous study of a different procedure for a different condition. In this study, Moseley et al²⁶ evaluated the use of arthroscopy to treat osteoarthritis of the knee and found that sham arthroscopy was as effective as real arthroscopy and that both were better than conventional treatment.

I was not long out of my orthopedic residency when this trial was published and was very aware of the debate that preceded it, as I once had to prepare a talk about it for resident rounds. I remember that there was a lively debate in the orthopedic community over the efficacy of the procedure before the results of this trial were released.

In contrast, the vertebral augmentation controversy had become a debate about the relative efficacy and the economics of specific techniques, not about the effectiveness of the entire concept. The mainstream had accepted the validity of the procedure, which was not the case in the knee arthroscopy trial.

In both vertebroplasty studies, the activetreatment groups and the sham-treatment groups all showed significant and rapid improvement in pain and disability, and these results were maintained over the study period. Though most vertebral compression fractures do heal, the clinical improvement is usually gradual over a period of weeks. This raises the possibility that the sham treatment was actualy an active placebo.

There is some evidence to support this possibility. In a randomized trial of the efficacy of selective nerve root blocks for lumbar radiculopathy, Riew et al²⁷ showed that injection with a local anesthetic alone, although not as efficacious as a local anesthetic plus a corticosteroid at allowing patients to avoid surgery, showed an effect long after the expected duration of the anesthetic. The effect persisted even at 5 years of follow-up.²⁸

Is it possible that the local anesthetic in this trial and the vertebroplasty trials acted as some sort of “reset button” for pain sensation? This is an area that may bear further investigation.

WHERE DOES THIS LEAVE US?

So where does this leave us? On one hand, randomized controlled trials comparing vertebral augmentation with conventional medical therapy^24,25 showed augmentation to be beneficial. On the other hand, the studies by Kallmes et al¹ and Buchbinder et al² indicate vertebroplasty is no more effective than sham surgery.

It is very difficult for me to look at my own experience with vertebral augmentation and say that, on the basis of these trials, I am no longer going to offer it to my patients. I understand on an intellectual level that these trials call the efficacy of the procedure into question, but on a visceral level I cannot rationalize it. When faced with a patient who is barely ambulatory or in fact bed-bound due to pain, my experience tells me that vertebral augmentation has a very high chance of getting them ambulatory within hours. The trials of vertebroplasty would indicate this is a placebo effect or that local anesthetic alone is as effective, but I am not yet ready to make that leap.

Cognitive dissonance seems to rule.