Science as a method of determining truth

When people hear the word "science" they often think about things like space exploration, computers, nuclear weapons, laboratory equipment, and medical advances. Much of this might better be called technology. There is also scientific knowledge, things taught in science classes, which it is fair to call science. In this document, however, we are concerned about science as a method for determining what is true, the so-called scientific method. This is not always the way scientists actually operate, but it is the way most scientists agree that they ought to operate. In this discussion we will use "science" mainly to refer to the ideal of the scientific method - the best known methods for determining what is true.  (Related essay: Is there something wrong with science?)

Scientific Truth

Reality as judge

The basis of science is to determine what is true, not by what the greatest authorities claim, or by what would be true in the ideal universe, or some magical revelation or inner feeling, but by what actually happens in the real world. Philosophers could theorize about whether a heavy object fell faster than a light object, but Galileo actually tried it  (contrary to expectations, both fell at the same speed). The way the world works is the ultimate measure of scientific truth, not the pronouncements of philosophers or other authorities.

The ability to correct false beliefs

Science recognizes that all beliefs have the possibility of error, so it is always open to new evidence. There is no dogma that must be believed, only principles that work well on the cases observed up until now. While some principles seem extremely reliable, they must be reconsidered when there is sufficient evidence that they don't work. A century ago it was believed that continents were firmly fixed in their positions and there was no way they could move. Over time evidence accumulated that they did gradually move - it was found that mountain ranges on both sides of the Atlantic had once been joined, and that seafloor spreading originated in faults at the bottoms of the oceans. As evidence accumulated, scientists changed their opinions, and now it is generally accepted that continents have moved considerably from their positions millions of years ago.  Because the scientific method provides for correcting faulty principles, the truth about continental drift was able to be recognized.

Is science precise, accurate, and authoritative?

Because we often associate science with technology and engineering, we often think of it being very rigid, precise, and authoritative. This isn't necessarily true. While science attempts to measure things as accurately as possible, it recognizes that every measurement has some inaccuracy associated with it. Often an error range is included when scientists or engineers describe a measurement. Science also deals with probabilities - a certain treatment makes it more likely for someone to recover from a disease, but there is usually no guarantee. While scientific experts may seem imposing and authoritative,  the good ones recognize that there are always limits on what they know.  Unlike many people in other professions, scientists usually discuss their findings cautiously, as in "Our research indicates that so-and-so might be the case, but more research is needed to confirm this."  This is in contrast to people promoting or selling things, who always speak as if they have total confidence.

One area in which good science has to be precise is in the way its claims are stated.  They must be clear and unambiguous. It should be clear what evidence would tend to confirm or deny any claim, and what we should expect to happen if the claim is true. The meaning of a scientific claim must not be a matter of judgment by the user.  If it were, the evidence might not support what the user thinks it does.

Need for firm foundations

Generally science deals with universal principles that might be of value to people anywhere the world and for many years in the future. The scientific knowledge we have now is essential as a foundation for making new discoveries later on. The penalty for false knowledge is great - many people can be affected and progress is impeded.

If, for example, it is believed that chewing on a certain kind of bark helps to reduce cold symptoms, scientists would be likely to spend a lot of time and money to find out just what component of the bark has this effect. This could lead to the development of better cold treatments that avoid various disadvantages of having to actually chewing on the bark. If it turns out that the original belief was wrong - that the bark was actually ineffective - then all the work to find the active ingredient would turn out to be wasted.

Because future progress depends on it, it is important that scientific knowledge be as reliable as possible.  The effort to make sure that scientific claims are accurate is one of the characteristics that separates science from more casual gathering of knowledge.

Why some sciences progress faster than others

Sciences can be categorized into different types.  Physical sciences include physics, chemistry, astronomy, and geology.  The physical sciences plus biology related science is often called natural science.  Science that deals with human activities is called social science.  Each kind of science has its own issues when it comes to  what has to be done to try to get better understanding.  In physical sciences like physics and chemistry, experiments are usually repeatable because there is a reasonably small number of factors that influence the result, like temperature, pressure, and the materials involved. Researchers create the conditions they are interested in and measure what happens.  Since they are usually aware of the things that influence the result, someone else repeating the experiment normally gets the same outcome.  If, say, chemical A and chemical  B  are mixed and heated to a certain temperature, they combine to form chemical C and release a certain amount of heat.  If someone else mixes the same amounts of each, they would expect to get the same results.

In medicine things get a lot more complicated.  Since every human body is different, there is no assurance that two people getting the same treatment will have the same response.  Even if the same person gets the same dose of the same drug for the same disease, they may not respond in the same way.  Their body may react differently because they have had the drug before, or because they are more tired, or because they have a more or less serious case of the disease or a different form of it or because of things they have eaten recently or any of thousands of other factors.  Unlike the chemistry situation, nobody can predict accurately how a patient will respond to any treatment.  It's still enormously useful to know that a certain treatment usually results in an improvement and rarely has bad side effects.  The difficulty comes from that fact that there are huge numbers of factors that affect the outcome.  These include not only the genetics of the patient, but many, many things that have happened to the patient over her lifetime that affect the condition of the body in some way.  Life experiences can be affected by culture, family traditions, education, and so on.

In social sciences like psychology or sociology or economics the situation is even more out of control.  All these depend on the human mind which, because it depends on learning over time, is far more affected by past experiences than bodies are.  Psychology experiments are very hard to design because the very fact that the subject is aware of being involved in the experiment is likely to change the way she reacts to things.  In economics and sociology many people are typically involved, so performing any sort of experiment may be impractical.  To add to the problem, ethical issues prevent performing many kinds of medical or social experiments because they may result in harm to the people involved.

Physical sciences may seem more "scientific" than medicine or social sciences because the results are more specific and over time there has been lots of opportunity to build on accurate knowledge acquired in the past.  Progress in medicine and social sciences seems far less clear, but this isn't because the people doing the work are less talented, but because the difficulties involved in getting reliable results are far greater.  I suspect that researchers in medicine and social sciences may sometimes be better critical thinkers because they have to deal with many issues about acquiring knowledge that never come up in physical sciences.

We'll often hear the phrase "the scientific method" to describe how scientists go about acquiring knowledge.  Because different scientific areas have such different properties, there is not really one scientific method.  In some fields it is normal to set up experiments and see what happens.  When humans are involved, care must be taken that people's knowledge of what's going on doesn't influence the result.  In areas like astronomy, it's impossible to try things out since we can't manufacture stars or planets in the lab.  The best that can be done is to try to devise models that agree with careful observations.  Some things, like making careful measurements, altering models when they don't work and publishing results, are necessary in all fields.

Scientists are fallible

Science as it is practiced in the real world doesn't always live up to the ideal of the scientific method.  Scientists are human and are subject to mistakes and inappropriate influences just like everyone else. Universities sometimes reward research faculty more on the basis of quantity of publishing than quality, resulting in a glut of papers that are rarely read. Technical papers may use a complex and difficult to understand writing style that creates the impression that the author is extremely knowledgeable but might result in reviewers accepting papers they don't fully understand. While scientists ideally would not let emotions get in the way of interpreting their data, often competition between them gets very intense and that may result  in researchers becoming attached to pet theories.  We need to recognize that individual scientists and  studies that they do can easily be mistaken.  We're on much firmer ground when there is a wide consensus among scientists, since it's part of their job to find out when their colleagues are wrong.  If dissenting voices are rare, then it's reasonable to have some confidence in what we're told.

Precise definition of a problem or hypothesis

In order be able to properly confirm or deny a scientific principle, it is necessary to define that principle in terms that are clear to proponent and skeptic alike. The claim "famous people die in groups of three" is impossible to test since different people can have different opinions about when a group starts or ends, or who should be considered famous. A possible way to improve this hypothesis might be to define a person as famous if their death is reported on the front page of at least half of the top ten newspapers in the United States. A new "group" could be started whenever a week goes by without any such deaths. At this point we would have a statement that can be tested (and will probably fail to be found accurate). Proponents could claim a different criterion, such as using the top ten newspapers worldwide instead of American ones and using ten days instead of a week to separate groups. The new hypothesis would have to be tested independently.

When a hypothesis is well stated, those who believe it and those who don't should at least agree on what actual observations would support the hypothesis and what would contradict it. We should also be able to determine how likely it is that the observed evidence might have occurred by chance.  After carefully collecting enough data, scientists should then be able to agree on whether the hypothesis is telling us something real about the world.

The null hypothesis: A is unrelated to B

It seems to be human nature to speculate about relationships between things. A basketball player who has just made several baskets is often assumed to have the "hot hand" and have a better chance to make his next shot. In other words, if he ordinarily made 48% of his shots, this would say he'd do better than 48% when the previous shot was a success, and worse than 48% when the previous shot was missed.  The "hot hand" hypothesis implies a relationship between shots the player has made in the recent past and shots made in the future.

There are many possible factors affecting future events, but the most common situation is that the two factors are not related to any important extent. The hypothesis that there is no relationship is called the "null hypothesis". In the case of the basketball player, the null hypothesis would say that the chances of his or her shots going in is not related to the success of previous shots, so for the above case, the chances would be 48% either way.

Predictions

Historical Observations

Some scientific hypotheses concern whether certain events occurred in the past. The theory that the universe began with a big bang, or that people evolved from ape-like ancestors, or that ancestors of Native Americans came to North America from Asia over a land bridge crossing the Bering Straits are of this type. It is hard to see these as predictive, since they concern things that happened long ago. Nevertheless, such statements do predict that certain kinds of evidence might be found in the future and certain kinds will not. Based on the big bang theory, scientists determined that a certain frequency of microwave radiation should be coming from all directions in the sky. This was later confirmed. The Theory of Evolution, as it is currently understood, predicts that more primitive fossils, such as those of dinosaurs, should be found in strata below, but not above, more advanced ones, like most mammals. So far this has been true. Theories of the origin of Native Americans might be verified by comparing their physical traits or cultural factors to see whether they are more similar to those of ancient Asians or ancient Europeans.

Correlation

If two factors are correlated it means that certain values of one make values of the other more or less likely. For example, a tall person is likely to weigh more than a short person, so height is correlated to weight. Of course there are exceptions, so height and weight are not perfectly correlated, but on the average a taller person will weigh more. This would be a positive correlation. A negative correlation means a greater quantity of one is associated with a smaller quantity of the other. Obesity is negatively correlated with life expectancy, since being more overweight is likely to result in living less long.

We can get a more quantitative idea about the relationship between occurrences by doing correlations. We might, for example, correlate the severity of punishment for a particular crime in various states with how frequently the crime is committed.

If there is a correlation between two factors, it may be that one was the cause of the other, but it may also be that both were influenced by a third factor. We might find that beer drinking is correlated with good health, but that a third factor, age, caused both. People in their twenties typically drink more beer than people in their sixties, and, being younger, are normally more healthy than people in their sixties. Despite the fact that drinking more beer is correlated with better health, it would be a mistake to conclude that drinking more beer makes people healthier.  We could also erroneously assume the reverse causation, as in the story of the person who noticed that the rooster crowed just before the sun came up, and deduced that the rooster caused the sun to rise.

Scientists will often use correlation to suggest the possibility that one thing causes another, but they can't prove causality using only correlation.

Experiments

When scientists can deliberately create a situation that tests some hypothesis, this is an experimental test. Some features that may be important if we are to trust experimental results are: controls, single and double blinding, proper statistical analysis, a published report, and replication. (more about experiments)

Confounding

Experiments typically try to determine how a change in one variable (say the amount of oat bran people eat) affects another quantity (like their blood cholesterol level). Suppose an experiment has one group of people eat an oat bran cereal for breakfast every day while a control group does whatever they want. After several months the oat bran group has lower average cholesterol levels. While the oat bran might be responsible, that group also may have had more milk because they ate the cereal with milk. Since both oat bran and milk were different between the two groups, it's possible that the milk rather than the oat bran might have caused the favorable result. The two variables, amount of oat bran and amount of milk, are said to be confounded, meaning we can't tell which is responsible for the result.

The placebo effect

People who believe they are being given an effective treatment for a medical problem are very likely to feel an improvement for their condition even if the treatment is worthless. This is called the placebo effect. It makes it very common for people to falsely jump to the conclusion that a treatment is effective even when it is not. In an ABC television special about people's beliefs, journalist John Stossel showed a class of college age students which had been given some pills that they were told might affect their sleep. Three quarters of the students said they had felt an effect and several of them were so convinced of their value that they were eager to get more of the pills. It turns out that the students were given pills which had no active ingredients.

Such pills or similar dummy treatments are known as "placebos". In experiments to test a real drug, the results for a group using the drug are compared with the results for a group using a placebo. If the only effect of the drug is the placebo effect, then the two groups should do about the same. If the drug is actually effective, the group taking the drug should do better.

Extraordinary claims require extraordinary evidence

An extraordinary claim is one that is highly improbable and inconsistent with known evidence, such as the claim that the tree in your front yard can talk. Even if a very reliable person told you this, you would be justified in assuming she was joking, lying, mistaken, or delusional rather than believing her.  All of these possibilities would be more likely than a talking tree.  If television news crews, established scientists and personal observation all confirmed that the tree could talk, then it might be reasonable to consider that the extraordinary had actually happened.