Doing Good Science

What are the methods of discovery, and how do we lessen the impact of our personal biases on our interpretation of the results?

© 2003 CAB International. Weed Ecology in Natural and Agricultural Systems (B.D. Booth, S.D. Murphy and C.J. Swanton)

While we like to think that scientists come to conclusions based on the cold, hard facts, this is not always the case. Rarely are the results of an experiment so compelling that there is only one possible interpretation. Even long-accepted 'facts' are sometimes revisited and discredited. For example, the extinction of the tambalacoque tree (Sideroxylon grandiflorum) has been linked to the extinction of the dodo bird (Raphus cucullatus). It was said that the seeds of this tree had to pass through the gut of the dodo to be able to germinate. When the dodo became extinct, tambalacoque seeds were no longer able to germinate. It made common sense. New evidence, however, has shown that this well-accepted 'fact' is not true. In fact, tambalacoque seedlings have been observed, although rare. The decrease in this species seems to have been caused by the introduced monkeys that consumed the fruit before the seeds were ripe and therefore they were unable to germinate when dispersed (Witmer and Cheke, 1991). Thus, common sense does not necessarily make good science.

The scientific method

To help avoid the pitfalls of relying on common sense, ecologists use the 'scientific method'. It focuses on asking good questions, making hypotheses, designing experiments to test them and then using this new information to re-evaluate our understand-

OBSERVATION observe spatial or temporal patterns

OBSERVATION observe spatial or temporal patterns

POSTULATE MODELS to explain the observations

NULL AND ALTERNATIVE HYPOTHESES based on model

DESIGN EXPERIMENT to test hypothesis

RETAIN NULL HYPOTHESIS predictions incorrect

REJECT NULL HYPOTHESIS refine alternative hypothesis

TEST HYPOTHESES using appropriate statistics

REJECT NULL HYPOTHESIS alternative hypothesis fully explains observations

Fig. 10.1. The scientific method.

ing of the world (Fig. 10.1). The first step in the scientific method is to make observations of ecological patterns. This sounds trivial, but science based on inaccurate observations is useless. From observations, we then propose models to describe or explain our observations. For example, if we observe that a weed species tends to be found only in open habitats, we might hypothesize that light is an important factor determining the species growth or survival. Next, we create one or several more specific hypotheses. For example, we might come up with the following two hypotheses to explain our observations:

• H1 - light has a direct positive effect on the growth of the weed and therefore it tends to be found in open habitats

• H2 - herbaceous insects that eat the weed prefer to live in shady areas, therefore only seedlings in open sunny locations survive

We also state the 'null hypothesis'. This is a statement that there is no effect expected. For our example our null hypothesis (Ho) is:

• Ho - light has no effect on the growth or survival of the weed.

For statistical reasons, we actually attempt to falsify our null hypothesis rather than 'prove' our hypothesis that there is an effect of light on growth or survival (Underwood, 1990). The reason this is done is similar to the legal ethic of being 'innocent until proven guilty'.

When we test hypotheses, there are two types of errors to be concerned with (Table 10.1). As type I and type II errors imply, the scientific method is not foolproof. Incorrect explanations about observations can result from mistakes at any point in the method. However, the scientific method does provide a common structure, and an opportunity to test whether what we perceive as fact is actually valid. Science is only as good as the investigator. So, how do we avoid bias and do good science? One way to do this is continually to reconsider what we believe to be true. Remember it was once common knowledge that the world was flat and that the sun revolved around the earth. These 'facts' once made sense and they were supported by the science of the day. Now, in light of our changing understanding of the universe, this seems ridiculous. Therefore, it is important to have an open and flexible mind.

Designing experiments

A hypothesis is tested using an experiment. There are numerous ways to design experiments, and many types of experiments can be used to test a single hypothesis. Experiments can be conducted either in environmentally controlled conditions such as in a greenhouse or a growth chamber, or in the 'field' where plants are grown and/or manipulated under natural or semi-natural conditions. The variables we are interested in are manipulated, and then we measure the response of the individual or population.

Table 10.1.

Type I and type II errors in hypothesis testing.

Error

Significance

Type I Type II

Null hypothesis is true, but rejected Null hypothesis is false, but not rejected

You mistakenly believe that there is a significant effect when there is not

You mistakenly believe that there is no significant effect when there is

Real effect

Statistical result

Null hypothesis true Null hypothesis false

Null hypothesis accepted Null hypothesis rejected

Correct Type II error Type I error Correct

To test our first hypothesis (H1) from above, we might grow plants in growth chambers where light levels are manipulated, but all other variables (e.g. temperature, moisture) are kept the same. The benefit of a lab-based study is that the researcher can control most of the environmental conditions, and therefore it is possible to isolate exact causes. However, such controlled conditions are not necessarily realistic; plants grown in greenhouses and growth chambers are not subject to natural variation in abiot-

Treatment 1 Treatment 2 Treatment 3

Full sun Partial shade Full shade

The mean height for each treatment is calculated using the mean height of each of the five replicates.

(19+18.3+20+19+17.3)/5 (17+17.3+17+16.7+15.7)/5 (15.7+15.3+14.3+16+16.7)/5 mean, = 18.7 mean2 = 16.7 mean3 = 15.6

Fig. 10.2. An example of an experiment testing whether light affects plant growth (height). The design consists of three treatments (full sun, partial shade, full shade) with five replicates each. Each data point is the mean height of the three individuals in each replicate. In cases where a plant dies (shown by an 'X') the mean of the remaining individuals is taken. Shown are the means and standard deviations (sd) for each treatment.

ic conditions such as temperature fluctuations. Alternatively, we could find a natural population of our weed species or plant seeds or transplants, and then reduce ambient light by adding shade cloth over some individuals. In such field experiments, environmental conditions are not tightly controlled (e.g. a severe storm could flood your plants); however, it is a more natural set-up for the plants and may be more reflective of ecological reality.

The two key features of an experiment are treatments and replication. Treatments are the number of types of manipulations made. For example, our field experiment might have three treatments: full (ambient) light, partial shade and full shade (Fig. 10.2). The treatment with full light is called the control because there is no experimental manipulation; the other treatments are compared to it see if there is an effect.

Every treatment is replicated several times; for example, we might set up each treatment five times. Replication is done for practical reasons. It ensures that your experiment is not ruined simply because one plant dies for reasons unrelated to treatments; for example, if someone steps on your plant. There are also statistical reasons for replication; there will be natural varia tion within a population, and replication allows us to accommodate this. If we had used one replicate plant per treatment, and the one in full light happened to have genes that caused it to be short, then we would have concluded erroneously that plants in shade grow taller. We would be unlikely to make this mistake if we have a range of plants growing in each treatment. Therefore, replication increases our ability to detect differences due to experimental treatments rather than differences due to natural variation. The number of replicates required will increase as natural variation increases. Of course, this is an extremely simple experimental design. Most experiments are much more complicated because they incorporate numerous types of treatments and their interactions.

Accuracy, precision and bias

When we collect data, we have to make sure they are accurate, precise and unbiased (Fig. 10.3). Accurate data closely reflect the true value of the variable (e.g. height, biomass, density) you are estimating. Precision describes how close the values of replicated data are to each other, and therefore is a

160 140 H

-E

120 -

<u

-E

100 -

80 -

60

-Q

TO

40

(0

>

20

0

true value of the population

Estimate a accurate precise not biased

0 0

Post a comment