Back to main page

Science as a method of determining truth

When people hear the word "science" they often think about things like space exploration, computers, nuclear weapons, laboratory equipment, and medical advances. Much of this might better be called technology. There is also scientific knowledge, things taught in science classes, which it is fair to call science. In this document, however, we are concerned about science as a method for determining what is true, the so-called scientific method. This is not always the way scientists actually operate, but it is the way most scientists agree that they ought to operate. In this text we will use "science" mainly to refer to the ideal of the scientific method - the best known methods for determining what is true.  (Related essay: Is there something wrong with science?)

Reality as judge

The basis of science is to determine what is true, not by what the greatest authorities claim, or by what would be true in the ideal universe, or some magical revelation or inner feeling, but by what actually happens in the real world. Philosophers could theorize about whether a heavy object fell faster than a light object, but Galileo actually tried it  (contrary to expectations, both fell at the same speed). Nature, not philosophers or authorities, is the ultimate judge of scientific truth.

The ability to correct false beliefs

Science recognizes that all beliefs have the possibility of error, so it is always open to new evidence. There is no dogma that must be believed, only principles that work well on the cases observed up until now. While some principles seem extremely reliable, they must be reconsidered when there is sufficient evidence that they don't work. A century ago it was believed that continents were firmly fixed in there positions and there was no way they could move. Over time evidence accumulated that they did gradually move - evidence was found that mountain ranges on both sides of the Atlantic had once been joined, and that seafloor spreading originated in faults at the bottoms of the oceans. As evidence accumulated, scientists changed their opinions, and now it is generally accepted that continents have moved considerably from their positions millions of years ago.  Because the scientific method provides for correcting faulty principles, the truth about continental drift was able to be recognized.

Is science precise, accurate, and authoritative?

Because we often associate science with technology and engineering, we often think of it being very rigid, precise, and authoritative. This isn't necessarily true. While science attempts to measure things as accurately as possible, it recognizes that every measurement has some inaccuracy associated with it. Often an error range is included when scientists or engineers describe a measurement. Science also deals with probabilities - a certain treatment makes it more likely for someone to recover from a disease, but there is usually no guarantee. While scientific experts may seem imposing and authoritative, it is reality, and not a person's credentials, that determines what is true.

One area in which good science needs to be precise is in the statements of its claims.  They must be clear and unambiguous. It must be clear what evidence would tend to confirm or deny any hypothesis, and what we should expect to happen if the statement is true. The meaning of a scientific claim must not be a matter of judgment by the listener.

Need for firm foundations

Generally science deals with universal truths that might be of value to people all over the world and for many years in the future. Often scientific knowledge is essential as a foundation for building new knowledge in the future. The penalty for false knowledge is great - many people are affected and progress is prevented.

If, for example, it is found that chewing on a certain kind of bark helps to reduce cold symptoms, scientists would be likely to spend a lot of time and money to find out just what component of the bark has this effect. This could lead to the development of better cold treatments that avoid various disadvantages of actually chewing on the bark. If it turns out that the original belief was wrong - that the bark was actually ineffective - then all the work to find the active ingredient would turn out to be wasted.

Because future progress depends on it, it is important that scientific knowledge be as reliable as possible.

Why some sciences progress faster than others

In hard (physical) sciences, like physics and chemistry, experiments are usually repeatable because there is a reasonably small number of factors that influence the result, like temperature, pressure, and the materials involved. In soft (social) sciences like psychology or sociology it is very difficult to establish reliable facts because there are so many relevant factors, like the culture of the people involved, family traditions, genetics, health, occupation, education, and peers. As a result it is usually much more difficult to find reliable general principles and devise foolproof tests, so these sciences tend to progress more slowly.

Bad science: jargon, obfuscation, priesthood

Science as it is practiced in the real world does not always live up to the ideal required by the scientific method. Universities sometimes reward research faculty more on the basis of quantity of publishing than quality, resulting in a glut of papers that are rarely read. It is common for such papers to use a complex and difficult to understand writing style that creates the impression of intellect while perhaps getting reviewers to accept it rather than admit they don't understand it. While there are legitimately some areas of science that cannot be understood without considerable background, it is also tempting for academics to use jargon and set themselves up as experts who are above questioning by those without credentials in their field. There may also be areas of research that have not yet sufficiently recognized the need for properly controlled experiments.

Precise definition of a problem or hypothesis

In order be able to properly confirm or deny a scientific principle, it is necessary to define that principle in terms that are clear to proponent and skeptic alike. The claim "famous people die in groups of three" is impossible to test since different people can have different opinions about when a group starts or ends, or who counts as famous. We can improve the hypothesis by defining a person as famous if their death is reported on the front page of at least half of the top ten newspapers in the United States. A new "group" could be started whenever a week goes by without any such deaths. At this point we would have a statement that can be tested (and will probably fail to be found accurate). Proponents could claim a different criterion, such as using the top ten newspapers worldwide instead of American ones and using ten days instead of a week to separate groups. The new hypothesis would have to be tested independently.

When a hypothesis is well stated, those who believe it and those who don't should at least agree on what actual observations would support it and what would contradict it. We should also be able to determine how likely it is that the observed evidence might have happened by chance.

The null hypothesis: A is unrelated to B

It seems to be human nature to speculate about relationships between things. A basketball player who has just made several baskets is often assumed to have the "hot hand" and have a better chance to make his next shot. In other words, if he ordinarily made 48% of his shots, this would say he'd do better than 48% when the previous shot was a success, and worse than 48% when the previous shot was missed.  This implies a relationship between shots the player has made in the recent past and shots made in the future.

There are many possible factors affecting future events, but the most common situation is that the two factors are not related to any important extent. The hypothesis that there is no relationship is called the "null hypothesis". In the case of the basketball player, the null hypothesis would say that the chances of his or her shots going in is not related to the success of previous shots, so for the above case, the chances would be 48% either way.

Predictions

Historical Observations

Some scientific hypotheses concern whether certain events occurred in the past. The theory that the universe began with a big bang, or that people evolved from ape-like ancestors, or that ancestors of Native Americans came to North America from Asia over a land bridge crossing the Bering Straits are of this type. It is hard to see these as predictive, since they concern things that happened long ago. Nevertheless, such statements do predict that certain kinds of evidence might be found in the future and certain kinds will not. Based on the big bang theory, scientists determined that a certain frequency of microwave radiation should be coming from all directions in the sky. This was later confirmed. The Theory of Evolution, as it is currently understood, predicts that more primitive fossils, such as those of dinosaurs, should be found in strata below, but not above, more advanced ones, like most mammals. So far this has been true. Theories of the origin of Native Americans might be verified by comparing their physical traits or cultural factors to see whether they are more similar to those of ancient Asians or ancient Europeans.

Correlation

If two factors are correlated it means that certain values of one make values of the other more or less likely. For example, a tall person is likely to weigh more than a short person, so height is correlated to weight. Of course there are exceptions, so height and weight are not perfectly correlated, but on the average a taller person will weigh more. This would be a positive correlation. A negative correlation means a greater quantity of one is associated with a lesser quantity of the other. Obesity is negatively correlated with life expectancy, since being more overweight is likely to result in living less long.

One can get a more quantitative idea about the relationship between occurrences by doing correlations. We might, for example, correlate the severity of punishment for a particular crime in various states with the frequency of occurrence of that crime.

If there is a correlation, it may be that one was the cause of the other, but it may also be that both were influenced by a third factor. We might find that beer drinking is correlated with good health, but that a third factor, age, caused both. People in their twenties typically drink more beer than people in their sixties, and, being younger, are normally more healthy than people in their sixties. We cannot conclude that it is the beer that make them healthy.  We could also erroneously assume the reverse causation, as in the story of the person who noticed that the rooster crowed just before the sun came up, and deduced that the rooster caused the sun to rise.

Scientists will often use correlation to suggest a hypothesis that one thing causes another, but they cannot prove causality is true using only correlation.

Experiments

When scientists can deliberately create a situation that tests some hypothesis, this is an experimental test. Some features that may be important if we are to trust experimental results are controls, single and double blinding, proper statistical analysis, a published report, and replication. (more about experiments)

Confounding

Experiments typically try to determine how a change in one variable (say the amount of oat bran people eat) affects another quantity (like their blood cholesterol level). Suppose an experiment has one group of people eat an oat bran cereal for breakfast every day while a control group does whatever they want. After several months the oat bran group has lower average cholesterol levels. While the oat bran might be responsible, that group also may have had more milk because they ate the cereal with milk. Since both oat bran and milk were different between the two groups, it is possible that the milk rather than the oat bran might have caused the favorable result. The two variables, amount of oat bran and amount of milk, are said to be confounded, meaning we can't tell which is responsible for the result.

The placebo effect

People who believe they are being given an effective treatment for a medical problem are very likely to feel an improvement for their condition even if the treatment is worthless. This is called the placebo effect. It makes it very common for people to falsely jump to the conclusion that a treatment is effective even when it is not. In an ABC television special about people's beliefs, journalist John Stossel showed a class of college age students which had been given some pills that they were told might affect their sleep. Three quarters of the students said they had felt an effect and several of them were so convinced of their value that they were eager to get more of the pills. It turns out that the students were given pills which had no active ingredients.

Such pills or similar dummy treatments are known as "placebos". In experiments to test a real drug, the results for a group using the drug are compared with the results for a group using a placebo. If the only effect of the drug is the placebo effect, then the two groups should do about the same. If the drug is actually effective, the group taking the drug should do better.

Extraordinary claims require extraordinary evidence

An extraordinary claim is one that is highly improbable and inconsistent with known evidence, such as the claim that the tree in your front yard can talk. Even if a very reliable person told you this, you would be justified in assuming she was joking, lying, mistaken, or delusional rather than believing her.  All of these possibilities would be more likely than a talking tree.  If television news crews, established scientists and personal observation all confirmed that the tree could talk, then it might be reasonable to consider that the extraordinary had actually happened.