Sections

Full review list

Search this site:

Guest editorial

Defining and providing essential evidence for practice

Evidence-based medicine (EBM) is, by its initial definition, a “new approach to teaching of the practice of medicine”.[1] Great efforts have thus been made to train clinicians to independently find, appraise, interpret, and apply evidence from original research reports, a process known as critical appraisal. This is an effective process for preparing the elite in EBM. However, most clinicians have a primary job of seeing patients and making care decisions, and, even if adept at critically appraising papers, they often have insufficient time to find and read research articles from scratch. Moreover, many are simply not interested in learning these advanced critical appraisal skills: despite considerable efforts, in 2006, only 3.6% of physicians in some tertiary hospitals in China knew what the Cochrane Library was, and only 8.6% could correctly interpret the number needed to treat.[2] Progress has been made in the development of an international appraisal system of healthcare interventions, but, as discussed at the recent Cochrane Colloquium held in Freiberg, Germany, the search for the best method to summarise and disseminate evidence for practice continues.

If the central practice of EBM is making care decisions consistent with current best evidence,[3] then searching and appraising original research articles is an important, but only preparatory, step. Obviously, not all the detailed information in the reports of primary studies and systematic reviews is relevant to decision making.[4] It is entirely possible that pertinent information in research reports can be determined, extracted, and distilled into a concise summary before it is provided to clinicians for application. The distilled information can be further integrated such that users would not have to search for diversely placed evidence for the same decision issue. If research evidence can be collectively identified, summarised, integrated and provided, clinicians would not need to find and appraise evidence from scratch.[5]

If not all the information in research reports is necessary for decision making, what information should be extracted and made available to clinicians and patients to aid in decision making? A piece of information should be necessary or essential, and can be considered such if, without it, a doctor or patient would find making a decision difficult or problematic. Any information that potentially would not influence or change any conclusion reached would be redundant.

Not all information required for decision-making needs to be generated from research. However, when making treatment decisions, answers to the questions below are essential, and are mostly provided by research:

1. What are the worst consequences if the disease is not treated? And how likely are such consequences to occur (prognosis or natural progress of disease)?
2. What outcome(s) can the treatment alter in a favourable manner?
3. How much can the treatment alter the outcome(s) (size of effect compared with placebo or another treatment and confidence interval)?
4. Is the size of the benefit consistent in various patient populations and care settings (homogeneity)?
5. How trustworthy are the answers to the above questions (internal validity determined by methodological quality)?
6. What is the probability that a particular patient will experience the size of benefit demonstrated for the treatment in research (external validity determined by patient and setting)?
7. Information on harms similar to questions 2–6.

Although knowledge on prognosis is important for any decisions on treatment, this factor is often neglected. If there is only a remote negligible risk for the patient, then choice of treatment may be considered inconsequential, whereas an immediate fatal risk always calls for urgent action. How likely is an average patient with hypertension to develop a major cardiovascular event in the next 5 years in the absence of any treatment? In China, many physicians thought the risk to be about 40%, and the general population estimated risk to be about 70%. Both these approximations are considerably higher than the research-based estimate of 8% for an average Chinese adult with hypertension.[2] This large over-estimation of risk is likely to lead to over-medication by the patient. The observation in China may reflect a wider tendency to disregard information on prognosis in the decision-making process.

The importance of the outcome and size of benefit for decision making is clear cut.[6] The outcome measured and the size of benefit are key in judging the clinical importance of treatment. The latter also facilitates assessment of the effectiveness of a particular treatment when compared with other interventions. Patients and physicians alike may have poor knowledge on both the important outcome(s) a treatment aims to alter and the effect size. For example, two thirds of Chinese residents thought that the ultimate goal of antihypertensive drugs was merely to lower blood pressure or alleviate symptoms, and were unaware of the more important long-term beneficial effect on lowering risk of stroke, and of coronary heart disease. When asked about the long-term effects of antihypertensive drugs, physicians estimated that antihypertensive drugs could reduce the 5-year risk of major cardiovascular events by 20% — considerably higher than the best research-based estimate of only 2% reduction for the same outcome.[2]

Practitioners of evidence-based medicine should be aware that there is always some degree of uncertainty about the actual size of the effect. If there is a wide confidence interval reported for the effect size, and/or heterogeneity among people enrolled, or settings of study, is large, it would be difficult to state with confidence the true size of effect, and caution should be exercised when making decisions.

Information generated from research reports can be useful if, and only if, it is trustworthy. The reliability of the results presented, commonly known as internal validity, is determined by the methodological quality of, or bias-reduction measures in, the study from which the information is derived. Obviously, strength of recommendations about a treatment should be proportionate to the strength of the internal validity of the report.

It should be considered that a large benefit for an important outcome, although described in more than one high-quality study, may not be repeatable in some patients. There are two major reasons for this. First, the benefit may truly vary among different patients and/or care settings. This may be particularly true when there is considerable heterogeneity among studies. Second, the intervention may be difficult to reproduce at a required standard: for example, surgical operations that require years of training and practice in order to reach the required level of surgical expertise. It may not always be possible to enlist a doctor of equivocal expertise to that of the doctor involved in the research setting. In both circumstances, the decision maker should examine and compare characteristics of the patients included in the study with those of the patient presenting for treatment (such as age, sex, severity, co-morbidities, and compliance), and characteristics of the care setting with those of available facilities (such as the standard of diagnosis, treatment equipment, and doctor’s expertise). Any notable differences between the definitive parameters of the research setting and the specific information (both in terms of the patient and resources) presented to the clinician could cast uncertainty about the size of benefit for a particular patient.

Information on the areas touched on in this Editorial is by no means all that is needed for making clinical decisions. We call for more research on what constitutes essential information for clinical decision making that can be identified from research. ACP Journal Club and Clinical Evidence are probably among currently the most far-sighted attempts of this kind. However, there is room for improvement. For example, information could be more consistently provided on the risk in the non-treated or placebo control group, the test results for heterogeneity and subgroup analyses, and important characteristics of patients and care setting.

For many interventions, high-quality evidence may be currently unavailable. But decisions still have to be made. For example, recommendations about alternative/complementary medicine are often based on long history of clinical use rather than randomised trials.[7] Such makeshift policies are often a result of the lack of high-quality research in the area[8] and should not be taken to mean that alternative/complementary medicine should always be practised at this standard. Recommendations will change when better evidence suggests otherwise.

Finally, it is impossible to conduct studies in all possible populations and settings. It is reasonable to suspect that our patients may fare differently from those in research settings if there is large unexplained heterogeneity or when it is difficult to reproduce treatment in a hands-on setting, or both. However, it is not justifiable to decline current best evidence simply by arguing that research from other countries or populations is not applicable to our specific patients or setting. Interventions shown to be effective in one population are often also effective in other populations. Until studies are carried in our own desired population and/or setting (which may never actually come about), evidence from other populations is the strongest foundation for clinical action.

In conclusion, until we can clearly define, sensibly integrate, and effectively disseminate the necessary research-derived information for use by millions of practising clinicians and policy makers, evidence-based medicine will remain a privilege for, and be of benefit to, a minority.

Jin-Ling Tang, MD, PhD
Professor and Director
Hong Kong Cochrane Branch
School of Public Health
The Chinese University of Hong Kong
Hong Kong, China

Shan Wang, MD
Professor and Director
People’s Hospital
Peking University Health Science Centre
Peking University
Beijing, China

• Subscribe: Formats | Institutional access | Monthly email alert
• Resources: Commissioned papers | EBM Tools | EBM links | Drug information | EBM training | Articles and Research
• About us: What we do | A guide to the text | Legal
• Contact us: By email | By post | By telephone
• Contribute: Introduction | Become a contributor | Become a peer reviewer

• Website Terms and Conditions |
• Terms and Conditions for purchase |
• Privacy policy |
• Home |
• Top