The Need for Concrete Improvement
in Abstract Quality

This section is compiled by Frank M. Painter, D.C.
Send all comments or additions to:    Frankp@chiro.org

Editorial ~ JAMA 1999 (Mar 24); 281 (12): 1129-1130

A Commentary on:

Accuracy of Data in Medical Abstracts of Published Research Articles
JAMA 1999 (Mar 24); 281 (12): 1110–1111

Margaret A. Winker, MD

---

The abstract is, aside from the title, the most frequently read and most easily accessed portion of an article reporting original biomedical research. The abstract provides an irreplaceable resource for busy clinicians, researchers, and authors searching for pertinent material in the source journal or in computerized databases. JAMA began publishing abstracts with articles on January 7, 1956, predating MEDLINE by a decade; other journals followed suit. 1 Structure was not added to JAMA abstracts until 1991 2 when the structured format developed by Haynes et al 3 and tested 4 and evolved 5 in Annals of Internal Medicine with the help of Edward J. Huth, MD, then Annals' editor, was adopted. Reading the abstract has never been a substitute for reading the article: crucial details of the study, such as patient selection and follow-up, definition of outcome measures, and study limitations, receive short shrift in the terse style of the abstract. 6, 7 A simple and straightforward abstract may obscure a more complex (and realistic) story within the text. These limitations aside, however, the abstract provides the reader with an efficient summary of the study that facilitates scanning many articles to find those that are the most pertinent to the reader's interests and needs.

The structure of abstracts is based on the components that are essential elements of teaching critical appraisal of the literature. ⁵ Structure appears to have improved the quality and usefulness of the abstract. 8-10 Of course, "quality" of an abstract depends on what the reader wants to know; one abstract format cannot serve the needs of all readers. 11, 12 However, the author can convey the key elements of the study, just as a journalist conveys the key elements of a news story: the abstract's sections Context, Objective, Design, Setting, Patients, Intervention, Main Outcome Measures, Results, and Conclusions should tell the reader why, what, how, where and when, who, what was done, what was measured, what was found, and what it means. Within these categories, certain information should be included that provides the reader with essential information and accurately reflects the material in the text. ⁵

Accurately reflecting the material in the text may seem the most basic requirement for an abstract. However, in this issue of THE JOURNAL, Pitkin and colleagues 13 find an astounding 18% to 68% of 264 abstracts in 6 large general medical journals Annals of Internal Medicine, BMJ, CMAJ, JAMA, Lancet, and New England Journal of Medicine had data in the abstract that were either inconsistent with or absent from the main body of the article. In addition to the astonishingly high number of deficiencies, the frequency of errors varied significantly from journal to journal. This result was especially troubling because abstracts are widely used, often separate from their text, as in MEDLINE and other databases, and data taken from the abstract may be reported and disseminated in other works, in other formats, and in the media.

When preliminary data demonstrating this problem were presented at the Third International Congress on Peer Review in Biomedical Publication in September 1997, along with the results of another study by Pitkin and Branagan ¹³ demonstrating that authors did not improve the quality of their abstract in response to specific instructions, JAMA took this study as a mandate to develop and implement abstract quality control procedures, which began with the January 1, 1998, issue. We developed quality criteria (Table 1), with a focus not only to improve accuracy, the bare minimum that should be expected of an abstract, but also to improve the quality of what was reported. Since then we have used these criteria to review and edit the abstracts of all articles accepted for publication.

The quality criteria were developed using evidence wherever possible, built on work of previous authors and supplemented by common sense. Item 1 reflects the importance of the structured abstract categories for reporting specific components of a study. ⁵ Items 2 and 3 are based on the findings of the study by Pitkin and Branagan. 14 Item 4, the years in which the study was conducted and the length of follow-up, provides the reader with the currency and scope of the study. Item 5 is included to improve reporting of negative as well as positive results. ⁵ Item 6 emphasizes that results should be quantified, preferably with confidence intervals, ⁵ because verbal expressions of frequency are interpreted differently by physicians, the general public, and physicians with different native languages. 15 Absolute differences rather than relative differences, item 7, provide the reader with a more accurate understanding of the effect of treatment for a population of patients. ⁵ Item 8 states that, for randomized trials, the intent-to-treat analysis should be specified and included so that results are not influenced by differential dropout rates. 16 For surveys, the response rate, item 9, is an important measure of quality. When potentially confounding factors have been controlled for in a model, they should be specified to enable the reader to determine whether uncontrolled confounding may remain (item 10). Finally, item 11 stipulates that conclusions should be consistent with the study results, and the stated implications of the study should be reasonably circumspecta subjective call, but a frequent debate.

Have our efforts improved abstract quality? A preliminary assessment of abstracts before and after the quality criteria were implemented was reassuring. While our baseline data were consistent with the disturbing results of the study by Pitkin et al ¹³ more than half of the 21 original research articles published in November 1997 had some discrepancy between the abstract and the text no discrepancies between the abstract and text were identified in the 27 articles published in November 1998 after the quality improvement step was instituted (J. C. Lantz, MLA, ELS, unpublished data, November 1998).

While the data presented should be accurate and consistent for every abstract, other components of the quality criteria are more subjective. We use our quality criteria as guidelines, and perfect adherence may not be achieved. Even with constant attention to abstract quality, since a primary goal of the abstract is to provide a concise summary of the study, the quality criteria sometimes must be compromised to prevent the abstract from duplicating the text.

Many important issues remain that have not been addressed by Pitkin et al or by our quality criteria. Are abstracts as accessible as they should be to both professionals and consumers? The Cochrane abstract guidelines aim to make abstracts as "readable as possible without compromising scientific integrity" (P. Middleton, BSc, written communication, February 11, 1999). Shouldn't medical journals take a similar tack? Why do discrepancies occur between the abstract and text? What form of abstract reporting is most helpful to physicians and others practicing evidence-based medicine? 17 Are different types of abstracts for a single article necessary to meet the differing needs of readers? And most important for editors and authors to address and researchers to evaluate, is essential information in the article missing from the abstract? ^{5, 12} While this question is the most difficult to assess, if the answer is yes, for the reader the abstract may cause more harm than good.

Regardless of the answers to these questions, authors, reviewers, and editors should pay increased attention to abstracts. The abstract must truly reflect the study, both in terms of specific data and overall message. After reviewing the abstract, the reader who does not have time to refer to the text should have an accurate impression of the study and should obtain useful information. Based on the findings of Pitkin and colleagues, ¹³ readers should be cautious and not assume that information reported in the abstract accurately reflects that in the text. However, the concrete responsibility belongs to editors and authors alone: they must improve the quality of abstracts to help ensure that studies achieve the maximum possible benefit for patients, physicians, and the research community.


Author/Article Information

 
Author Affiliation: Dr Winker is Deputy Editor of JAMA.
 
Corresponding Author and Reprints: Margaret A. Winker, MD, JAMA, 515 N State St, Chicago, IL 60610 (e-mail: margaret_winker@ama-assn.org). Editorials represent the opinions of the authors and THE JOURNAL and not those of the American Medical Association.

Acknowledgment: I thank Jane C. Lantz, MLA, ELS, for her thoughtfulness and diligence in the pursuit of better abstracts.

REFERENCES

1. Abstracts.
N Engl J Med.
1968;278:48.

2. Rennie D, Glass RM.
Structuring abstracts to make them more informative.
JAMA.
1991;266:116-117.
MEDLINE

3. Ad Hoc Working Group for Critical Appraisal of the Medical Literature.
A proposal for more informative abstracts of clinical articles.
Ann Intern Med.
1987;106:598-604.
MEDLINE

4. Huth EJ.
Structured abstracts for papers reporting clinical trials.
Ann Intern Med.
1987;106:626-627.
MEDLINE
5.
Haynes RB, Mulrow CD, Huth EJ, Altman DG, Gardner MJ.
More informative abstracts revisited: a progress report.
Ann Intern Med.
1990;113:69-76.
MEDLINE

6. Froom P, Froom J.
Deficiencies in structured medical abstracts.
J Clin Epidemiol.
1993;46:591-594.
MEDLINE

7. Narine L, Yee DS, Einarson TR, Ilersich AL.
Quality of abstracts of original research articles in CMAJ in 1989.
CMAJ.
1991;144:449-453.
MEDLINE

8. Taddio A, Pain T, Fassos FF, Boon H, Ilersich AL, Einarson TR.
Quality of nonstructured and structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association.
CMAJ.
1994;150:1611-1615.
MEDLINE

9. Trakas K, Addis A, Kruk D, Buczek Y, Iskedjian M, Einarson TR.
Quality assessment of pharmacoeconomic abstracts of original research articles in selected journals.
Ann Pharmacother.
1997;31:423-428.
MEDLINE

10. Wilczynski NL, Walker CJ, McKibbon KA, Haynes RB.
Preliminary assessment of the effect of more informative (structured) abstracts on citation retrieval from MEDLINE.
Medinfo.
1995;8(pt 2):1457-1461.
MEDLINE

11. Spitzer WO.
The structured sonnet.
J Clin Epidemiol.
1991;44:729.

12. Heller MB.
Structured abstracts: a modest dissent.
J Clin Epidemiol.
1991;44:739-740.
MEDLINE

13. Pitkin RM, Branagan MA, Burmeister LF.
Accuracy of data in abstracts of published research articles.
JAMA.
1999:281:1110-1111.

14. Pitkin RM, Branagan MA.
Can the accuracy of abstract be improved by providing specific instructions? a randomized controlled trial.
JAMA.
1998;280:267-269.
MEDLINE

15. Nakao MA, Axelrod S.
Numbers are better than words: verbal specifications of frequency have no place in medicine.
Am J Med.
1983;74:1061-1065.
MEDLINE

16. Newell DJ.
Intention to treat analysis: implications for quantitative and qualitative research.
Int J Epidemiol.
1992;21:837-841.
MEDLINE

17. Naylor CD, Williams JI, Guyatt G.
Structured abstracts of proposals for clinical and epidemiological studies.
J Clin Epidemiol.
1991;44:731-737, 741-742.
MEDLINE

Return to EVIDENCE–BASED PRACTICE