‘Any process at any stage of inference which tends to produce results or conclusions that differ systematically from the truth’.
(Adapted from Murphy. The Logic of Medicine. Baltimore: John Hopkins University Press. 1976.)
One of the key steps in evidence based medicine, and sadly one that is often forgotten, is to appraise evidence before making a decision as to whether to use the it. Skills in critical appraisal involve a systematic evaluation of research in order to establish, amongst other things, whether the results of a particular study are valid both internally (reflecting how well the study was conducted) and externally (reflecting how generalizable the results are). Internal validity is determined by the extent to which the study is free from bias. As the risk of bias increases, so does the likelihood that the results a particular study do not reflect the truth.
So what types of bias should we be looking for?
For randomized controlled trials, the Cochrane handbook defines a common classification scheme for sources of bias, summarized below:
The handbook also points out other potential sources of bias, such as in the study design (e.g. a carry over effect from a cross-over RCT) or a “contamination” effect whereby participants in one arm, adopt the intervention in the other.
But does all this searching and assessment for bias make a real difference?
Simply put…yes it does.
Take “Reporting bias” for example – differences between reported and unreported findings, which itself incorporates several types:
Empirical evidence has shown that trials showing positive findings are more likely to be published, and often quicker, than trials with negative findings.
So imagine this – you are doing a systematic review of clinical trials, on a healthcare question relevant to your patient or population. You find your relevant included studies, you systematically analyze, summarize and find a clear benefit. Great.
How would you feel though if after all the time and effort you put in to do your review, your results were totally hampered by bias and you never even realized it? – because you only had access to published trials which were more likely to show a positive finding.
But wouldn’t it be nice to have a consensus agreement that every clinical trial done should be registered and every result within it reported? Think how much stronger your review would be. Think how much stronger the clinical decisions we make that affect patient care would be.
If your answer is yes (and it should be) then you should have a look, then sign up to the AllTrials campaign, because that’s exactly what they have been calling for.
I will be writing more about bias shortly.