ESP bank: Science: The scientific method of enquiry

Introduction – an anecdote What is the scientific method?  Sources of error Can we apply the scientific method in everyday life? Is the scientific method always applicable? Can we fool ourselves? And who is fooling whom? References Quiz and answers Thinking points Notes to Thinking points

Introduction – an anecdote

The physicist Richard P Feynman suggested that it was ‘such a scientific age that we have difficulty in understanding how witch doctors could ever have existed’¹. But he went on to describe a Pacific island population which benefited from the cargo planes that would land on the US army airstrips. After the war, when western priorities changed, they wondered how they could make things just like before. The answer was obvious: put a line of fires along a ‘runway’, with a man in a hut close by, and get him to wear half a coconut on each ear just as the airmen had done with their headphones. But the planes and their food supplies did not return.

The islanders pondered how to solve this dilemma. Change the shape of the coconuts? Choose a more senior tribesman to sit in the ‘control tower’? Feynman coined the phrase ‘cargo cult science’ to describe people who follow all the apparent precepts and forms of scientific investigation – but they are missing something essential.

This is the scientific method of enquiry.

What is the scientific method?

The scientific method has four stages²:

1. Observe a phenomenon
2. Formulate a hypothesis that is consistent with your observations
3. Use this hypothesis to predict an outcome
4. Test your predictions by experiment

If your observations are consistent with your hypothesis, you have a theory. Conversely, if there is a discrepancy, a scientist must be prepared to modify or discard his or her hypothesis.

The scientific method can be represented like this:

An example of a hypothesis would be, ‘A spider builds a web to catch prey’. From this, we might predict that destroying its web would prevent it from catching food.

When the hypothesis is known to have limited validity, we make a model. An example of this would be the wave model of light. It successfully explains phenomena such as diffraction, but fails to describe the photoelectric effect. It may continue to be applicable within a limited range of measurable parameters.

A scientific theory or law results from a hypothesis which has been confirmed through repeated experimental tests. The ‘laws of nature’ suggests they can be universally applied.

Sources of error

Error in experiments has several sources:

• error intrinsic to the instrument
• systematic error

Intrinsic error could be due to the accuracy of the measuring device. It can produce a measurement higher or lower than the ‘true’ value, so it is called random error.

A non-random error (or systematic error) biases the result in one direction. This could be due to incorrect calibration of an instrument, or a non-representative survey sample.

We have developed standard ways of estimating (and in some cases reducing) error. This information should also be given in a report so that it can be taken into account.

A measurement without quoted error is meaningless.

Can we apply the scientific method in everyday life?

People who eat well with plenty of fruit and vegetables in their diet tend to live longer, healthier lives. Green leafy vegetables are high in antioxidants, but claims that antioxidant tablets are a useful supplement are based on observational studies rather than interventional studies (13).

Take the recent case of the immunisation vaccine in Britain for the diseases measles, mumps and rubella (MMR). A paper published in the respected medical journal, The Lancet, in 1998 identified a relationship between chronic enterocolitis in children and neuropsychiatric dysfunction (9).

The press confused association with causality (7), and the resultant panic led many parents to avoid giving their children the vaccine. In six of the eight cases that showed the symptoms of autism, the interval was so short (a week or less) that any direct or indirect link is highly implausible. The result, however, has been outbreaks of measles that otherwise would not have occurred.

In a highly unusual move, the paper was formally retracted, saying, ‘We wish to make it clear that in the paper no causal link was established between MMR vaccine and autism as the data were insufficient’.

What of other drivers of public opinion? Dr Stephen Ladyman, then representing the Department of Health in Britain, said, ‘It might not be possible to absolutely prove that greenhouse gas emissions cause global warming, but the evidence suggests they do, and so wise governments act accordingly’ (10).

‘If people understood [the scientific] principle’, he continued, ‘then the government could far more clearly make decisions based upon the weight of evidence without waiting until ‘clear proof’ emerges. Governments could even change their position when the evidence changes without being accused of U-turns or incompetence.’

Is the scientific method always applicable?

You are unlikely to step off a skyscraper on the assumption you will survive because ‘Gravity is only a theory’.

However, in social interactions between people, you may not be able to isolate phenomena or repeat a measurement. For example, a lawyer may argue a case in court, but she cannot try again if she fails.²

Are dreams outside the scientific method? After all, they seem irrational and personal experiences. Sigmund Freud established symbols such as snakes and staircases, and connected them to childhood and sexual needs. He diagnosed his students by applying a reductive method (6). He wanted to reduce the meaning of dreams to a series of symbols, like a language. CG Jung, a former pupil of Freud, would propose a constructive method. He thought the imagination to construct a story was more important. But, according to the Journal of Abnormal Psychology, both authors abuse the language analogy but that both are equally logical.

Nevertheless, psychologists would say that a dream is an associative reaction to a sensation. This is a valid hypothesis that can be tested, for example, by scratching the ear of a sleeping subject.

Can faith be treated differently? A hypothesis must be open to ‘objective empirical test, at least in principle’ (7). Science is a self-correcting process where a hypothesis can be tested. ‘I believe in God’ is a statement, not a hypothesis. However, ‘All things in the universe were created by God in the six days of creation as described in Genesis 1:1 – 23’ is. To test it, though, you first have to define ‘God’.

The scientific community is aware that opinion changes. This has been seen in studies of continental drift, erosion, DNA, anthropology, astronomy and quantum physics. People who support ‘intelligent design’, however, freely admit that they will retain their belief at all cost.

But we can still ask the question, ‘Why does person A believe in statement B?’ A part of the brain called the ‘temporal lobe’ is associated with religious experience. Stimulation in this region with weak electromagnetic fields induces a ‘sensed presence’ (12) – the feeling that someone is there. Dr Chris French of the Anomalous Psychology Research Unit at London’s Goldsmith’s College suggests that if phenomena such as astrology have no basis in fact, we can still use the scientific method to explain why people believe in them. (11)

Can we fool ourselves? And who is fooling whom?

Much criticism was directed this year towards the South Korean biologist Hwang Woo-Suk for falsifying results on stem cell research. But how could an entire team of experienced scientists maintain the deceit? Richard Feynman tells the story of the struggle to measure the charge on an electron. In 1910, Robert Millikan calculated the value slightly inaccurately, and a number of teams repeated the experiment (in a process known as peer review) to try to achieve a more precise figure. ‘If you plot them as a function of time,’ Feynman noted, ‘you find that one is a little bigger than Millikan’s, and the next a little bigger than that.’ Finally they settle on a higher number still. Why didn’t they arrive at the correct number immediately or have randomly higher or lower results? Could it be that when they got a value they considered too high, they looked for reasons to discredit it? And when it was closer to Millikan’s value, they ‘did not look so hard’?

Is false logic to blame for bogus claims? Many people believe that ‘carrots help you see in the dark’, for example. While it is true that vitamin A deficiency causes night blindness, we cannot necessarily infer the reverse. (The myth may lie in the campaigns by European governments during the Second World War to encourage the population to eat the cheap and plentiful vegetable.)

Or was the South Korean claim more similar to cigarette advertising where, if you say your product is low on nicotine, the assumption is that it must be better for you than a rival product? Never mind if it is high on tar – or that all cigarettes are bad for you. Good news is more likely to attract funding.

The ‘superbug’ MRSA (methicillin-resistant staphylococcus aureus) is at epidemic proportions in British hospitals, so the government was happy to claim a ‘drop of 6% in the last six months by comparison with the same figures last year’ (5). Opponents say that figures covering six months rather than twelve are misleading.

In short, all data must be handled in the same way in an atmosphere of open communication.

Complementary therapies such as acupuncture are based on a series of diverging and seemingly contradictory pseudo-scientific models. However, an individual may report some improvement in her condition. Is this a result of the treatment? Or would the patient have got better anyway? Perhaps something else is at play here: the placebo effect. In controlled tests, a chemically ineffective pill can provide equally beneficial effects if the patient is unaware she has not been prescribed the actual medicine.

The fact that the pill contains nothing but sugar does not negate the statistical validity of the test – nor does it undermine the value of the recovery.

References

1 Cargo Cult Science by Richard P Feynman, 1974. http://calteches.library.caltech.edu/51/2/CargoCult.htm
2 http://teacher.pas.rochester.edu/phy_labs/AppendixE/AppendixE.html
3 Cutting stress may increase chances of pregnancy, The Guardian, 21 June 2006
4 based on http://physics.ucr.edu/~wudka/Physics7/Notes_www/node6.html
5 Guardian Unlimited / Today’s Issues / Q&A: Complementary therapies, drug trials, MRSA & other killer superbugs
6 The Journal of Abnormal Psychology, ed Prince, Morton, pp.369 -383
7 This baby suffered brain damage and epilepsy after the MMR jab, (www.dailymail.co.uk)
8 Science Methods, Misuse and Madness (www.physics.gmu.edu)
9 Why can’t doctors be more scientific? Hugh Pennington (www.lrb.co.uk)
10 www.spiked-online.com
11 www.gold.ac.uk/psychology/staff/french
12 Bring me a God helmet, and bring it now, The Guardian, 17 June 2006
13 Food for thought on alternative therapies, The Guardian, 24 June 2006

Quiz

1. What’s the difference between:
a. a hypothesis
b. a model
c. a law

2. A study of deaths in 1918 due to the global influenza pandemic is:
a. observational
b. interventional

3. The link between lung cancer and passive smoking is:
a. causal
b. associative

4. If an error is intrinsic to the measuring device, it is:
a. random
b. non-random. 

Quiz answers

1. a. is a tentative idea without experimental proof
b. is restricted to explaining a given phenomenon, and its shortcomings are recognised
c. uses observations and  is held to be universally true³

2. Observational. It would be impossible to intervene in a past event.

3. Associative. It would be as impossible to trace the cause specifically as it would be to subject a person to a controlled test. Never the less, the link between lung cancer and smoking has long been established, and there is overwhelming evidence to associate it with secondary inhalation.

4. Random. It is equally likely to be higher or lower than the true value.

Thinking points

Discuss the following. See the notes below for some ideas:

1. Does each theory build progressively on the preceding one?

2. Does the burden of proof lie with the person making the claim?

3. Should the National Health Service pay for ‘complementary therapies’ such as homeopathy or reflexology despite the lack of scientifically rigorous evidence? (see section 5 )

4. Is it necessary / acceptable to test drugs on humans to see if the results are consistent with the predictions? (see sections 1 & 4))

5. People in the 15th century believed the sun revolved around the earth. What evidence did they have to support that hypothesis? (see section 1)

6. Is the minority opinion always wrong?

7. ‘We get ill because of advances in medicine’ seems a strange hypothesis. How can this be rationalised?

8. Can astrology (rather than astronomy) be tested according to the scientific method? (see section 4)

9. You are probably learning English right now. Can the scientific method be applied to learning a language?  (see section 4)

Notes to Thinking points

1. Look at the ideas of Popper, Kuhn and Feyerabend on Wikipedia, the online self-edited encyclopaedia (https://en.wikipedia.org)

2. There is a rigorous process of peer review (see ‘Can we fool ourselves?’). If you are prepared to subject yourself to it, and present all the data in a spirit of open cooperation, you can expect constructive criticism from the scientific community.

3. In Britain, doctors can refer patients for alternative therapies free of charge. But there is no inherent right to such treatment, and it is up to the doctor to decide what is appropriate. The case may be viewed differently in palliative cases (where the patient is going to die), and the emphasis is on treatment, not cure. Some studies have shown positive results that could save money: psychotherapy lowered stress levels in 80% of women whose menstrual cycle had been disrupted. It could therefore be a viable alternative to expensive and often complex fertility treatment (3).

4. In March 2006, six men were taken seriously ill after taking part in a medical trial at a private research unit. There was debate about whether the drug had been successfully tested on animals previously. Another theory was that the drug functioned differently in humans than it did in animals.

5. They would have said, ‘Look! The sun rises in the east and sets in the west’ – just as it would according to their hypothesis. Famously, Galileo Galilei was tortured by the Vatican for his refusal to deny his belief in the Copernican theory of 1543. His other astronomical observations (such as the orbits of the known planets and comets) showed him that the Earth rotated around the Sun.

6. Clearly not, as the case of Galileo (see above) shows. But the following train of thought highlights the dangers of poor logic (8): 1. Everyone loves an underdog. 2. Everyone thought Galileo was wrong. 3. Everyone thinks I am wrong. 4. I might be right too. (see section 3)

7. ‘Ministers claim that the rise in infection rates [for MRSA in hospitals] is linked to advances in medicine, requiring more operations which can lead to infection.’ (5) (see section 5)

8. Yes. In fact, it has. Over 60 occupations were traced to test the hypothesis, ‘The astrological sun sign influences your career’. The results showed that people in all careers were equally likely to be born at any time of the year (7).

9. Whether it is a mother tongue or foreign language, we can come to recognise the vocabulary and structures of a language when they are repeated. But Ebbinghaus (6) made it clear that the causal connections are complicated: ‘When a train enters a large station, there are many paths over which it might pass; but its actual path depends on the position which was given to switches immediately before the train’s arrival’.