Bias – alive and kicking

Definition of bias

‘Any process at any stage of inference which tends to produce results or conclusions that differ systematically from the truth’.

(Adapted from Murphy. The Logic of Medicine. Baltimore: John Hopkins University Press. 1976.)


One of the key steps in evidence based medicine, and sadly one that is often forgotten, is to appraise evidence before making a decision as to whether to use the it.  Skills in critical appraisal involve a systematic evaluation of research in order to establish, amongst other things, whether the results of a particular study are valid both internally (reflecting how well the study was conducted) and externally (reflecting how generalizable the results are).  Internal validity is determined by the extent to which the study is free from bias. As the risk of bias increases, so does the likelihood that the results a particular study do not reflect the truth.

So what types of bias should we be looking for?

For randomized controlled trials, the Cochrane handbook defines a common classification scheme for sources of bias, summarized below:

sources of bias


The handbook also points out other potential sources of bias, such as in the study design (e.g. a carry over effect from a cross-over RCT) or a “contamination” effect whereby participants in one arm, adopt the intervention in the other.

Many researchers use the Cochrane tool for assessing the risk of bias in reviews, which has generally been found to be very acceptable by users.

But does all this searching and assessment for bias make a real difference?

Simply put…yes it does.

Take “Reporting bias” for example – differences between reported and unreported findings, which itself incorporates several types:

reporting bias

Empirical evidence has shown that trials showing positive findings are more likely to be published, and often quicker, than trials with negative findings.

So imagine this – you are doing a systematic review of clinical trials, on a healthcare question relevant to your patient or population. You find your relevant included studies, you systematically analyze, summarize and find a clear benefit. Great.

How would you feel though if after all the time and effort you put in to do your review, your results were totally hampered by bias and you never even realized it? –  because you only had access to published trials which were more likely to show a positive finding.

The Cochrane handbook describes ways of addressing reporting biases, such as searching for data from multiple sources, including grey literature and searching for unpublished studies.

But wouldn’t it be nice to have a consensus agreement that every clinical trial done should be registered and every result within it reported?  Think how much stronger your review would be. Think how much stronger the clinical decisions we make that affect patient care would be.

If your answer is yes (and it should be) then you should have a look, then sign up to the AllTrials campaign, because that’s exactly what they have been calling for.

I will be writing more about bias shortly.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s