The problem of false balance when reporting on science

Peter Ellerton, The University of Queensland

How do you know the people billed as science experts that you see, hear and read about in the media are really all that credible? Or have they been included just to create a perception of balance in the coverage of an issue?

It’s a problem for any media and something the BBC’s Trust is trying to address in its latest report on science impartiality in programming.

As part of ongoing training, staff, particularly in non-news programs, were told that impartiality is not just about including a wide range of views on an issue, as this can lead to a “false balance”. This is the process of providing a platform for people whose views do not accord with established or dominant positions simply for the sake of seeming “balanced”.

The BBC has been criticised before for “false balance” and there are reports now that certain climate change sceptics are banned from BBC News, although this is denied by the BBC.

It’s understandable that such false balance could grow from a desire to seem impartial, and particularly so since public broadcasters such as the BBC and the ABC in Australia are sensitive to claims of imbalance or bias.

Couple this with the need to negotiate the difficult ground of expert opinion, authentic balance and audience expectation, not to mention the always delicate tension between the imperatives of news and entertainment, and it hardly seems surprising that mistakes are made. An investigation this year found the ABC breached its own impartiality standards in its Catalyst program last year on statins and heart disease.

Finding the right balance

How then can journalists decide the best way to present a scientific issue to ensure accurate representation of the views of the community of experts? Indeed, how can any of us determine if what we are seeing in the media is balanced or a misrepresentation of expert opinion?

Hard to find the right balance.
Flickr/Paxson Woelber , CC BY

As I have written elsewhere, it is important to not confuse the right to be heard with an imagined right to be taken seriously. If an idea fails to survive in the community of experts, its public profile should diminish in proportion to its failure to generate consensus within that community.

A common reply to this is that science isn’t about consensus, it’s about the truth. This is so, but to use a consensus as evidence of error is fallacious reasoning.

While it’s true that some presently accepted notions have in the past been peripheral, the idea that simply being against the majority view equates to holding your intellectual ground in the best tradition of the enlightenment is ludicrous.

If all views are equal, then all views are worthless.

Were I to propose an idea free of testing or argument, I could not reasonably expect my idea to be as credible as those subject to rigorous experimentation and collaborative review. If such equality did exist then progress would be impossible, since progress is marked by the testing and rejection of ideas.

Defining an expert

In the case of science, this testing is the process of experimentation, data analysis and peer review. So if someone – scientist or otherwise – has not worked and published in an area, then they are not an expert in that area.

The first imperative for a journalist covering any story is to determine exactly in what field the issue best sits and then to seek advice from people who work and publish in that field.

Knowing how the issue fits into the broader picture of scientific investigation is very useful in determining this. It is one of the reasons that good science journalism follows from having journalists with some training in science.

Such a selection process, performed transparently, is an excellent defence against charges of bias.

Avoiding false balance

False balance can also be created by assuming that a person from outside the field (a non-expert) will somehow have a perspective that will shed light on an issue, that the real expert is too “caught up in the details” to be objective.

But suggesting that an expert is naive usually indicates an attempt at discrediting rather than truth seeking. Credibility is more about process than authority, and to be a recognised expert is to work within the process of science.

Also, if a piece of science is being criticised, we should ask if the criticism itself has been published. It’s not enough that someone with apparent authority casts doubt as this is simply an appeal to authority – an appeal that critics of mainstream science themselves use as a warrant to reject consensus.

A second journalistic imperative would be to recognise that not all issues are binary.

Coins may have two sides but not so every science issue.
Flickr/monkeyc net, CC BY-NC-SA

The metaphor that a coin has two sides is a powerful one, and the temptation to look at both sides of an issue is naturally strong. But the metaphor also assumes an equal weighting, and that both sides present the same space for discussion.

Proof and evidence

When an issue is genuinely controversial, the burden of proof is shared between opposing views. When a view is not mainstream, say that scientists are engaged in a conspiracy to defraud the public, the burden of proof sits with those promoting that view.

In such cases, as Christopher Hitchens succinctly put it:

What can be asserted without evidence can also be dismissed without evidence.

Attempting to dishonestly shift the burden of proof is a common device in the push to have young earth creationism taught in science classrooms.

The idea of “teaching both sides” or that students should be allowed to make up their own minds seems again like a recourse to the most basic ideas of a liberal education, but is in reality an attempt to bypass expert consensus, to offload the burden of proof rather than own it.

The fact is, that for issues such as creationism, vaccination and that climate change is occurring and is a function of human activity, it’s not about journalists suppressing views, it’s about quality control of information.

Stay with the issue

A classic means of muddying the waters is to employ straw man arguments, in which the point at issue is changed to one more easily defended or better suited to a particular interest. Politicians are adept at doing this, dodging hard questions with statements like “the real issue is” or “what’s important to people is”.

An expert versus who?

Deniers of climate science often change the issue from global warming to whether or not consensus is grounds for acceptance (it alone is not, of course), or focus on whether a particular person is credible rather than discuss the literature at large.

The anti-vaccine lobby talks about “choice” rather than efficacy of health care.
Young earth creationists talk about the right to express all views rather than engage with the science. Politicians talk about anything except the question they were asked.

The third imperative, therefore, is to be very clear as to what the article or interview is about and stick to that topic. Moving off topic negates the presence of the experts (the desired effect) and gives unsubstantiated claims prominence.

The impartiality checklist

The best method of dealing with cranks, conspiracy theorists, ideologues and those with a vested interest in a particular outcome is the best method for science reporting in general:

• insist on expertise
• recognise where the burden of proof sits
• stay focused on the point at issue.

If the media sticks to these three simple rules when covering science issues, impartiality and balance can be justifiably asserted.

Correction: This article was amended on July 17, 2014 to include a report of the BBC’s denial that a climate change sceptic was banned from the public broadcaster.

Peter Ellerton, Lecturer in Critical Thinking, The University of Queensland

This article was originally published on The Conversation. (Reblogged by permission). Read the original article.

 

The Rejection of Expertise

by Tim Harding

(An edited version of this essay was published in The Skeptic magazine,
September 2015, Vol 36 No 3 p.36,  titled ‘Who needs to Know?’ It has since been republished in the Australian Doctor magazine 30 October 2015. The essay is based on a talk presented to the Victorian Skeptics in May 2015 ).

Anti-vaccination campaigner, Meryl Dorey is on record as saying that we should ‘do our own research’ instead of accepting what the doctors and other qualified experts tell us.  Seasoned skeptics will be aware that ‘Do your own research!’ is a common retort by cranks and conspiracy theorists to those who dare to doubt their claims.  It is a convenient escape hatch they use when trying to win a debate without the bothersome burden of providing their own evidence.

Of course, what they mean by this exhortation is not to do any actual scientific or medical research.  It takes a bit of tertiary education in the relevant field to be able to do that.  For them, ‘research’ means nothing more than googling for less than an hour on the Internet. They naively equate such googling with the years of study and experience it takes to become a qualified expert.  Their message is that anybody with internet access can become an instant but unqualified expert on anything.  Or worse still, that expertise doesn’t even count – all opinions are equal.

The reality is that googling is a notoriously unreliable source of information – there are sound reasons why Wikipedia is not allowed to be cited as a source in university assignments.  The problem is that without expertise in the field in question, few googlers are capable of knowing which sources are reliable and which aren’t.  Anything found on the internet becomes ‘knowledge’.  Mere opinions become ‘facts’.

Another problem is that googlers are often unaware of the wider knowledge context of the specific pieces of information they have found on the internet. In contrast, experts are as much aware of what they don’t know as what they do know.  As Professor Stephan Lewandowsky of the University of Bristol puts it:

‘Here is the catch: to know how much more there is to know requires knowledge to begin with.  If you start without knowledge, you also do not know what you are missing out on.’

This paradox gives rise to a famous result in experimental psychology known as the Dunning-Kruger Effect. Named after Justin Kruger and David Dunning, it refers to a study they published in 1999. This study found that people who lack the knowledge or wisdom to perform well are often unaware of this fact. This is almost more dangerous than complete ignorance, because unlike Donald Rumsfeld, they don’t even know what they don’t know.

Professor Tom Nichols, a US national security expert wrote last year about the ‘death of expertise’; a Google-fueled, Wikipedia-based, blog-sodden collapse of divisions between professionals and amateurs, teachers and students, knowers and wonderers – between those with any expertise in an area and those with none at all.  He sees this situation as not only a rejection of knowledge, but also the processes of knowledge acquisition – a rejection of science and other pursuits of rationality.

Nichols is particularly critical of otherwise intelligent people who are ‘doing their own research’ on the internet and second-guessing their doctors by refusing to vaccinate their children, leading to an entirely avoidable resurgence of dangerous infectious diseases such as whooping cough and measles.

So how did it all come to this sorry state of affairs?  I think that there are basically four contributing factors: the blurring of facts and opinions; a misunderstanding of democracy; a misunderstanding of the Argument from Authority; and the dissipation of media accountability.  I will now discuss each of these factors in turn and then outline some benefits of listening to experts.

Blurring facts and opinions

According to the Stanford Encyclopaedia of Philosophy, a fact is a state of affairs that is the case.  The usual test for a statement of fact is verifiability; that is, whether it can be demonstrated to correspond to experience.  Scientific facts are verified by repeatable careful observation or experiment.  In other words, a fact is that which makes a true statement true.  For instance, the statement ‘It is raining’ describes the fact that it actually is raining.  The rain that falls can be objectively measured in a rain gauge – it is not just a matter of opinion.

On the other hand, an opinion is a judgment, viewpoint, or statement about matters commonly considered to be subjective, such as ‘It is raining too much’.  As Plato said: ‘opinion is the medium between knowledge and ignorance’.

The last few decades have seen the growth of a postmodernist notion that truth is culturally relative and that all opinions are equal.  What’s worse is a gradual blurring of the important distinction between facts and opinions.  A disturbing feature of the public debate about climate change is the confusion between science and policy.  Because they conflict with some political policies, there is a tendency for the findings of climate scientists to be treated as ‘just another opinion’.  This is a marked change from a few decades ago, when the findings of epidemiologists about the links between smoking and cancer were widely accepted as facts rather than opinions.

Misunderstanding democracy

Reducing the influence of experts is sometimes mistakenly described as ‘the democratisation of ideas’.  Democracy is a system of government – it is not an equality of opinions.  Whilst the right of free speech prevents governments from suppressing opinions, it does not require citizens to treat all opinions equally or even take them into account.  Equal rights do not result in equal knowledge and skills.  As Professor Brian Cox has said:

Deakin University philosopher Dr. Patrick Stokes has argued the problem with ‘I’m entitled to my opinion’ is that it has become shorthand for ‘I can say or think whatever I like’ without justification; and that disagreement is somehow disrespectful.  Stokes suggests that this attitude feeds into the false equivalence between experts and non-experts that is an increasingly pernicious feature of our public discourse.

Professor Michael Clark of LaTrobe University gives an example of a public meeting recently, when a participant asked a question that referred to some research, a senior public servant replied: ‘Oh, everyone has a scientific study to justify their position, there is no end to the studies you could cite, I am sure, to support your point of view.’  Clark describes this is a cynical statement, where there are no absolute truths and everyone’s opinion must be treated as equally valid.  In this intellectual framework, the findings of science can be easily dismissed as one of many conflicting views of reality.

Misunderstanding the Argument from Authority

A common response from cranks and conspiracy theorists (and even some skeptics) to citations of expertise is ‘that’s just the argument from authority fallacy’.  Such a response ignores the obvious fact that all scientific papers and other forms of academic writing are chock full of citations of experts.  The notion that the written outputs of the world’s universities and scientific institutions are all based on a logical fallacy is preposterous.  Anybody who thinks that has clearly not thought through the implications of what they are saying.

The Argument from Authority is often misunderstood to be a fallacy in all cases, when this is not necessarily so.  The argument becomes a fallacy only when used deductively, or where there is insufficient inductive strength to support the conclusion of the argument.

The most general form of the deductive fallacy is:

Premise 1: Source A says that statement p is true.

Premise 2: Source A is authoritative.

Conclusion: Therefore, statement p is true.

Even when the source is authoritative, this argument is still deductively invalid because the premises can be true, and the conclusion false (i.e. an authoritative claim can turn out to be false). This fallacy is known as ‘Appeal to Authority’.

The fallacy is compounded when the source is not an authority on the relevant subject matter. This is known as Argument from false or misleading authority.

Although reliable authorities are correct in judgments related to their area of expertise more often than laypersons, they can occasionally come to the wrong judgments through error, bias or dishonesty. Thus, the argument from authority is at best a probabilistic inductive argument rather than a deductive argument for establishing facts with certainty. Nevertheless, the probability sometimes can be very high – enough to qualify as a convincing cogent argument. For example, astrophysicists tell us that black holes exist. The rest of us are in no position to either verify or refute this claim. It is rational to accept the claim as being true, unless and until the claim is shown to be false by future astrophysicists (the first of whom would probably win a Nobel Prize for doing so). An alternative explanation that astrophysicists are engaged in a worldwide conspiracy to deceive us all would be implausible and irrational.

An artist’s depiction of a black hole

As the prominent British environmental activist Mark Lynas has said ‘…if an overwhelming majority of experts say something is true, then any sensible non-expert should assume that they are probably right.’

Thus there is no fallacy entailed in arguing that the advice of an expert in his or her field should be accepted as true, at least for the time being, unless and until it is effectively refuted. A fallacy only arises when it is claimed or implied that the expert is infallible and that therefore his or her advice must be true as a deductive argument, rather than as a matter of probability.  Criticisms of cogent arguments from authority can actually be a rejection of expertise, which is a fallacy of its own.

The Argument from Authority is sometimes mistakenly confused with the citation of references, when done to provide published evidence in support of the point the advocate is trying to make. In these cases, the advocate is not just appealing to the authority of the author, but providing the source of evidence so that readers can check the evidence themselves if they wish. Such citations of evidence are not only acceptable reasoning, but are necessary to avoid plagiarism.

Expert opinion can also constitute evidence and is often accepted as such by the courts.  For example, if you describe your symptoms to your doctor and he or she provides an opinion that you have a certain illness, that opinion is evidence that you have that illness. It is not necessary for your doctor to cite references when giving you his or her expert opinion, let alone convince you with a cogent argument. In some cases, expert opinion can carry sufficient inductive strength on its own.

Dissipation of media accountability

I have no doubt that the benefits of the internet generally outweigh the costs.  However, there are some downsides that need be considered rather than just glossed over.  An obvious negative is the decline of newspapers and competent professional journalism.  Specialist science or medical journalists are a rarity these days.  Generalist journalists often get their science stories wrong, or engage in misleading false balance – the equating of professional expertise with amateur ignorance.

Another problem is the blurring of the distinction between journalism and blogging – and I say this as a blogger myself.  Unlike bloggers, journalists are subject to professional standards and editorial control. Some bloggers are anonymous, which removes their accountability to even their own readers for the accuracy of what they write.

There is a risk that when non-experts google, they are inclined to give equal weight to information from both professional journalists and amateur bloggers, regardless of its reliability and accuracy.

Benefits of expertise

Whilst experts are human and can mistakes, they have a pretty good batting average compared to laypersons.  The advice that experts provide is far more likely to be true than advice from non-experts in the field in question.  This has obvious benefits for society as a whole, for example in terms of public health and safety, environmental protection and managing the economy.  There are good reasons why we don’t let amateurs design aircraft, bridges and tall buildings.  But there are also some major benefits for the individual in listening to advice from experts as opposed to non-experts.

For instance, if you trust your doctor, you’re actually more likely to do better when you’re sick, according to a study recently published by General Hospital Psychiatry.  This study, of 119 people with either breast, cervical, intestinal or prostate cancer, found that from three months following diagnosis, those patients who did not trust their doctors were not only more distressed but also more physically disabled.  They were less likely, for example, to be able to go for long walks or take care of themselves.  Patients who felt anxious about being rejected and abandoned suffered the most from not trusting their doctors.

Trusting your doctor has clear health benefits. You’ll be more likely to try new drugs, follow your treatment plan (jointly agreed with your trustworthy doctor), share important medical information, take preventative measures (e.g. screening) and have better-controlled diabetes and blood pressure.

Up to half of the failures in treatment reported by patients are due to not following the regime suggested by doctors.  This increases the risk of hospitalisation and extended ill health.  Another study at the University of California has found a small but statistically significant association between how much patients trusted their doctors and how much their symptoms improved within two weeks (allowing for different factors that could have influenced the outcome).

As Professor Michael Clark has said, people who use Dr. Google to diagnose their symptoms before visiting an actual doctor, sometimes ask to be tested for diseases they do not have, or waste time seeking a second opinion because they are convinced that their ‘research’ has led them to a correct diagnosis. If it were really that easy, would doctors have to spend all those years in medical school? Prof. Clark has also said that:

“Using Google to find the answer to Trivial Pursuit questions is not the same as researching a complex question. Experts do have skills and one of those is the ability to use high quality sources, up to date theoretical frameworks, and critical thinking based on their experience in a particular field. This is why an expert’s answers are going to be more accurate and more nuanced than a novice.”

References

Clark, M., and Lawler, S., ‘Why we need to listen to the real experts in science’.  The Conversation. January 1, 2015.

Harding, T., ‘Argument from authority’. The Logical Place. June 23, 2013.

Hinnen et al. ‘Lower levels of trust in one’s physician is associated with more distress over time in more anxiously attached individuals with cancer’. Gen Hosp Psychiatry. 2014 Jul-Aug;36(4):382-7.

Lewandowsky, S., and Pancost, R., ‘Are you a poor logician?‘ Logically, you might never know’. The Conversation. November 6, 2014.

Nichols, T., ‘The Death Of Expertise’. The Federalist, January 17, 2014.

Stokes, P., ‘No, you’re not entitled to your opinion’. The Conversation. October 5, 2012.

Thom, D.H., et al. ‘Measuring Patients’ Trust In Physicians When Assessing Quality Of Care’.  Health Aff (Millwood), University of California. 2004 Jul-Aug;23(4):124-32.

Further reading

Nichols, T., ‘The Death Of Expertise’. Oxford University Press, 2017.

Copyright notice: © All rights reserved. Except for personal use or as permitted under the Australian Copyright Act, no part of this website may be reproduced, stored in a retrieval system, communicated or transmitted in any form or by any means without prior written permission. All inquiries should be made to the copyright owner, Tim Harding at tim.harding@yandoo.com, or as attributed on individual blog posts.

If you find the information on this blog useful, you might like to consider making a donation.