Reanalysis of data suggests that effect is not a placebo effect

EDITORIrnich et al reported that acupuncture was superior to massage though not to sham acupuncture for neck pain.1 This suggests that acupuncture is effective but that this is due to a placebo effect.

The statistical method used (comparing improvements in pain between groups with pairwise t tests) is of questionable efficiency. Firstly, regression analysis including baseline score as a covariate has greater statistical power than comparison of change. 2 3 Secondly, each pairwise comparison in a three group trial ignores one third of the patients; such comparisons are thus underpowered when compared with regression modelling of all data.

Analysis of change scores, such as that reported, favours the group with worse baseline pain scores (in this case, the group that had sham acupuncture) because of regression to the mean4; conversely, analysis of follow up scores alone favours the group with less baseline pain. Regression analysis gives similar results regardless of the direction of baseline imbalance.

Irnich kindly provided me with raw data for reanalysis. To compare the effects of treatment on pain score one week after treatment (the prespecified primary outcome measure) I undertook a linear regression analysis. The covariates used were baseline score; treatment group; and several diagnostic variables (somatisation, depression, history of trauma, pain localisation, pain site (neck/other), pain type (relieved by heat/not relieved by heat), concomitant symptoms, neurological findings, and diagnosis (myofascial/other)).

Treatment was coded as two dummy variables: use of any acupuncture technique and use of true acupuncture. Acupuncture, sham laser acupuncture, and massage were thus coded 1, 1; 1, 0; and 0, 0 respectively. This analysis estimates the effects of acupuncture needling and placebo effects of acupuncture independently. Backwards stepwise regression was used where a P value of 0.05 was the criterion for keeping a variable in the model. Analyses were conduced on Stata 6 (College Station, Texas).

Depression, baseline score, and use of true acupuncture remained in the final regression model. The interpretation is that acupuncture needling is of benefit in neck pain and that this is not attributable to a placebo effect. Patients receiving true acupuncture had improvements in pain (adjusted for baseline score and presence of depression) of 11.5 points (95% confidence interval 3.5 to 19.5 points; P=0.005) more than those in the massage and sham groups. When the analysis was restricted to patients who received either sham laser or true acupuncture, acupuncture led to a reduction in pain score (adjusted for baseline pain) of 9.4 points greater than sham laser (0.9 to 18.0 points; P=0.031). These results differ substantially from those reported in the original paper.

From BMJ.com