Response to Evidence Review of Back Pain Therapies
by Cherkin Et Al

By Anthony L. Rosner, Ph.D., LL.D. (Hon.)

June 30, 2003


Published back-to-back in the same issue of the Annals of Internal Medicine with the flawed meta-analysis by Assendelft et al. [1] is a "review of the evidence" by Cherkin et al. for the management of back pain by a number of physical interventions, including acupuncture, massage therapy, and spinal manipulation. [2] From the point of view of a reviewer basing his perspective from the literature pertaining to spinal manipulation, this overview of treatment alternatives is unfortunately highly problematic. In striving to "provide clinicians, patients and health plans with a clear and balanced understanding of the current evidence" pertaining to the use of these CAM (complementary and alternative medical) therapies, this paper regrettably offers precisely the opposite.

Most perplexing is its arbitrary inclusion and exclusion of studies to arrive at its conclusions. Although the authors' original intent was to cite systematic reviews and original articles describing the results of RCTs published since the reviews were conducted, there are too many examples of

(i)   qualifying RCTs that were excluded,

(ii)   RCTs of dubious quality that were included,

(iii)   descriptions of pertinent RCTs with no further indication why or how they were either included or excluded, and

(iv)   the omission of earlier systematic reviews or meta-analyses with inadequate explanations of their shortcomings or flaws — or even how the current study built upon these earlier investigations. It is almost as if one were witnessing revisionist history.

More specifically, some of the shortcomings in Cherkin's paper can be summarized as follows:


1.   Failure to resolve conflicting previous reviews:

In an effort to filter out low-quality studies, rating systems of trial quality have abounded as an attempt to assure that the edifice of evidence used to warrant a therapeutic approach is more than a house of cards. These form the cornerstone to both systematic literature reviews and meta-analyses, the former defined as a comprehensive and rigorous review of the peer-reviewed scientific literature requiring a predetermined threshold of graded quality in order to be included. In meta-analyses, on the other hand, actual effect sizes are calculated from pooled results of different clinical trials using a variety of statistical procedures and taking into account the size of each study.

A meta-analysis by one of the authors of the current study (Shekelle) published in the Annals of Internal Medicine, retrieved 58 articles representing 25 trials and supported the short-term benefit of spinal manipulation in some patients, particularly those with uncomplicated, acute low back pain. Data regarding chronic low-back pain at the time of this publication were judged insufficient to evaluate the efficacy of spinal manipulation in managing this particular condition. [3]

Several years later, this qualification no longer stood. The rise in the stature of evidence supporting the use of spinal manipulation in managing chronic low-back pain could be described as meteoric. In a systematic review of 16 randomized controlled trials involving manipulation, van Tulder and his colleagues singled out two of high-quality. [4, 5] Van Tulder's review explicitly stated that evidence supporting manipulation for chronic low-back pain is found to be actually stronger than that for acute conditions ("There is limited evidence that manipulation is more effective than a placebo treatment for acute LBP [level 3]. There is no evidence that manipulation is more effective than [other] physiotherapeutic applications . . . or drug therapy. . . . There is strong evidence that manipulation is more effective than a placebo treatment for chronic LBP [level 1]. There is moderate evidence that manipulation is more effective for chronic LBP than usual care by the general practitioner, bed-rest, analgesics, and massage [level 2])". [6]

A somewhat differing interpretation was reached in Bronfort's systematic review. [7] Here, the evidence supporting spinal manipulation for managing either acute or chronic low-back pain was judged to be "moderate," while that for a mix of chronic and acute low-back pain was considered "inconclusive." Furthermore, all but one of the back pain studies considered to be of sufficient validity were eliminated by the criteria invoked by van Tulder, while the latter study included one trial [5] that had been rejected by Bronfort.

Yet another systematic review of randomized clinical trials cites adequate follow-up periods, avoidance of cointerventions, and avoidance of dropouts as frequent strengths. Recurrent weaknesses, however, include randomization procedures, sample sizes, and blinded assessments of outcomes—the latter being virtually impossible to perform in a trial involving manual therapy. [8] Finally, a meta-analysis of 51 literature reviews of spinal manipulative therapy suggests that, although the overall methodologic quality was low, 9 of the 10 methodologically best reviews reached positive conclusions regarding spinal adjustments. [9]

The fact that systematic reviews may conflict in both their conclusions and as to which studies to accept is admittedly troublesome. None of these inconsistencies, conflicts or resolutions were ever brought to light by the current authors as to why their most recent review should be accepted as more definitive than past similar efforts.


2.   Exclusions of previous studies:

In terms of outcomes, there is no apparent reason why the highly positive findings of van Tulder [6] or Giles [10] were not integrated into the body of the authors' discussion concerning spinal manipulation. To further illustrate the inconsistency and arbitrariness of this review, one need only wonder why the Giles study, in particular, was mentioned only in context of the acupuncture literature—in which neither back dysfunction nor pain significantly improved. In the context of spinal manipulation, in which significant improvements in both criteria were noted, there is no mention of any kind of this study in Cherkin's paper.

In reviewing cost-effectiveness, the authors argue that they have only chosen "the few effectiveness RCTs that measured cost" in order to minimize bias that might have been experienced with observational data. It then perplexes this reviewer why they entirely overlooked a trial concerned with lumbar disc herniation (a subject which the authors introduced only in connection with cauda equina syndrome). This particular orphaned investigation examined 40 patients with unremitting sciatica diagnosed as due to lumbar disc herniation with no clinical indication for surgical intervention. Subjects were randomized into two treatments: (i) chemonucleolysis (chymopapain injection under general anesthesia) and manipulation (15-minute treatments over 12 weeks, including soft tissue stretching, low-amplitude passive maneuvers of the lumbar spine and the judicious use of side-posture manipulations). Back pain and disability were appreciably lower in the manipulated group at 2 and 6 weeks with no improvement or deterioration in the chemonucleolytic group. By 12 months there were improvements in both groups with a tendency toward superiority in the manipulated cohort. Costs of treatment in the manipulated group were less than 30% of the total costs encountered by the injected patients; furthermore, the latter group averaged expenditures of £300 for treatment failures with no such costs experienced by the manipulated population. [11]


3.   Inclusion of study of questionable quality:

In discussing cost-effectiveness and spinal manipulation in the context of RCTs, Cherkin cites only his own study in which any advantages of spinal manipulation in terms of outcomes or cost-effectiveness in comparison to physical therapy (McKenzie treatments) or the use of an educational booklet were not evident. [12] The shortcomings of this study, including its misrepresentation of therapies and the overgeneralization of results, are so extensive that they have been presented elsewhere in no less than four separate venues. [13-16] In fact, the deficiencies of its design are so severe to have merited a special review by the Royal College of General Practitioners—which concluded that this particular RCT neither adds nor detracts from the evidence base regarding appropriate interventions for low back pain. [17] Since this review can only be as robust as the literature upon which it rests, one would have to closely scrutinize a review in which the lead author has chosen his own RCT whose inadequacies may have neutered its clinical utility.

CONCLUDING REMARKS:

From the perspectives of some of the recent literature addressing spinal manipulation, this review of outcomes and cost-effectiveness contains several glaring inconsistencies and omissions which seriously undermine its credibility. It falls far short of the mark in providing the "balanced" perspective originally sought by the authors and demanded of reviews of this nature. To summarize, Cherkin's review falls short of achieving its objectives due to:

• Its failure to resolve conflicts in previous literature reviews;

• Its inexplicable omissions of previous and highly pertinent studies; and

• The author's citation of only one of his own, widely discredited papers as the sole source of data regarding cost-effectiveness and spinal manipulation.

1. Assendelft WJJ, Morton SC, Yu EI, Suttorp MJ, Shekelle PG.
   Spinal manipulative therapy for low back pain: A meta-analysis of effectiveness relative to other therapies.
   Annals of Internal Medicine 2003; 138: 871-881.
   

2. Cherkin DL, Sherman SJ, Deyo RA, Shekelle PG.
   A review of the evidence for the effectiveness, safety, and cost of acupuncture, massage therapy, and spinal manipulation for back pain.
   Annals of Internal Medicine 2003; 138: 898-906.
   

3. Shekelle PG, Adams AH, Chassin MR, Hurwitz EL, Brook RH.
   Spinal manipulation for low-back pain.
   Annals of Internal Medicine 1992; 117: 590-599.
   

4. MacDonald RS, Bell CM.
   An open controlled assessment of osteopathic manipulation in nonspecific low-back pain.
   Spine 1990; 15: 364-370.
   

5. Sanders GE, Reinert O, Tepe R, Maloney P.
   Chiropractic adjustive manipulation on subjects with acute low-back pain: Visual analog scores and plasma beta-endorphin levels.
   Journal of Manipulative and Physiological Therapeutics 1990; 13: 391-395.
   

6. van Tulder MW, Koes BW, Bouter LM.
   Conservative treatment of acute and chronic nonspecific low back pain: A systematic review of randomized controlled trials of the most common interventions.
   Spine 1997; 22 (18): 2128-2156.
   

7. Bronfort G:
   Efficacy of manual therapies of the spine.
   Amsterdam, 1997, Thesis Publishers.
   

8. Koes BW, Bouter LM, van der Jeijden GJMG.
   Methodological quality of randomized clinical trials on treatment efficacy in low back pain.
   Spine 1995; 20: 228-235.
   

9. Assendelft WJJ, Koes BW, Knipschild PG, Bouter LM.
   The relationship between methodological quality and conclusions in reviews of spinal manipulation.
   Journal of the American Medical Association 1995; 274(24): 1942-1948.
   

10. Lynton G. F. Giles; Reinhold
    Chronic Spinal Pain Syndromes: A Clinical Pilot Trial Comparing Acupuncture,
    a Nonsteroidal Anti-inflammatory Drug, and Spinal Manipulation
    J Manipulative Physiol Ther 1999 (Jul); 22 (6): 376–381
    This is the first of 3 articles in Giles brilliant Chronic Spinal Pain review

11. Burton AK, Tillotson KM, Cleary J.
    Single-blind Randomised Controlled Trial of Chemonucleolysis and Manipulation
    in the Treatment of Symptomatic Lumbar Disc Herniation
    European Spine Journal 2000 (Jun); 9 (3): 202–207
    

12. Cherkin DC, Deyo RA, Battie M, et al.
    A Comparison of Physical Therapy, Chiropractic Manipulation, and Provision
    of an Educational Booklet for the Treatment of Patients with Low Back Pain
    New England Journal of Medicine 1998 (Oct 8); 339 (15): 1021-1029

13. Rosner AL.
    Evidence-based clinical guidelines for the management of acute low back pain:
    Response to the guidelines prepared for the Australian Medical Health and Research Council.
    Journal of Manipulative and Physiological Therapies 2001; 24(3): 214-220.
    

14. Freeman MD, Rossignol AM.
    A critical evaluation of the methodology of a low-back pain clinical trial.
    Journal of Manipulative and Physiological Therapies 2000; 23(5): 363-364.
    

15. Chapman-Smith D.
    Back pain, science, politics and money.
    The Chiropractic Report November 1998; 12(6).
    

16. Rosner A.
    Fables or Foibles: Inherent Problems with RCTs
    J Manipulative Physiol Ther 2003 (Sep); 26 (7): 460–467

17. Royal College of General Practitioners.
    Unpublished update of CSAG guidelines (reference 12; 1999.
    

18. Rosner A.
    Response to Meta-Analysis Published By Assendelft Et Al.
    www.FCER.org, posted 07/09/03.

Return to LOW BACK PAIN