Theory and Hypothesis
We’re winding down to the year-end holidays and CRM news is slow so I decided to scratch an itch. I’ve been interested in the difference between a theory and a hypothesis for a long time but lately it seems we’ve done a bang up job of confusing the two, not because of anyone’s evil intentions but due to differing definitions. So I decided to do some research and to the best of my ability see if the confusion could be dispelled. Of course, some people might like the confusion or prefer their definitions but this is simply my attempt to provide some organization.
The difference between hypothesis and theory is a lot like the difference between data, information, and knowledge, which I have been writing about all year. Data, information, and knowledge represent a hierarchy of, among other things, truth though some scholars like the early data scientist, the late Claude Shannon, would have also said that they represent increasing entropy in the top down view i.e. knowledge to data.
By analogy a theory is a hypothesis that has had some work done. Specifically, in the formal sense, a theory has been scientifically tested and found to be valid for explaining certain aspects of the world. The trouble comes from realizing that theory has both formal and colloquial definitions and that people sometimes use them interchangeably as they do data and information.
Lawyers have theories of their cases like Col. Mustard, in the library, with the candlestick. In a case tried in court the opposing attorney might concede the library and the candlestick but dispute the actor in this case the colonel offering instead another suspect.
Here’s where it gets interesting. The prosecutor accusing Col. Mustard has to prove the theory beyond doubt but the defense only has to offer a plausible alternative in other words, to sow enough doubt that the prosecutor’s case falls apart. The arbiters of this pseudo scientific experiment are twelve people on the jury. This is pretty close to scientific inquiry but it falls short in part because while the theory of the case might be proven or disproven, there are too many variables such as the jurors to compare a trial to a scientific experiment. After all juries get it wrong often; witnesses’ recollections can’t always be trusted, and so on.
The scientific definition of theory is more rigorous. A scientific theory is testable at any time and is the product of testing a hypothesis and finding that the hypothesis does, indeed, explain some aspect of the natural world. Experiments are designed so that they can prove or verify a hypothesis or disprove or falsify it. Once arrived at, a theory is one of the highest forms of scientific knowledge on a par with a scientific law, like gravity.
Remember the Col. Mustard example above—Mustard, candlestick, library? A theory describes the mechanism from which we get the conclusion and that’s the important difference between a theory and a law. A scientific law does not posit a mechanism to explain phenomena. Take gravity for example. The law of universal gravitation has been around for a long time. Newton provided the math for it saying basically that gravity was inversely proportional to the distance between two objects and directly proportional to their masses. But the mechanism of gravitation is still being worked out at places like Cern where scientists smash atomic nuclei together to observe the subatomic particles that result hoping to discover the particles that cause gravity.
Netting this out we know gravity exists from repeated observation even if we don’t exactly know how. At my house, my wife simply expects the car to run, I would say she believes in the law of the car. I attempt to understand the car’s mechanisms so that I can understand if a repair is necessary. I think of myself as believing in the theory of the car.
Lastly for this section, laws and theories have to be falsifiable meaning you must be able to design experiments that will conclusively prove or disprove them. For a very long time Newton’s explanation of physics was thought to be the last word because it seemed provable in all tests. But then along came Einstein who showed that Newton’s laws were a special case of a more generalized law that took into account the speed of light and what happens to matter as it approaches that universal speed limit.
Now this is important. More often than is healthy some of use conflate the meaning of theory. Is it the colloquial theory that’s more like a hypothesis that we’re talking about or is it the scientific and verifiable through experimentation concept that scientists hold to? The answer is important because we can often see arguments about theory that seem insoluble like the participants are talking past one another. One might say “It’s just a theory” implying the “it” is simply a hypothesis while another might say, it’s a theory and it’s scientifically proven.
Everything goes kaput if someone brings a deity into the discussion. Something done by a deity might be quite nice but completely incapable of proof. Quick—design an experiment that tests the impact of a deity. The first thing you’d have to do is figure out a way to control for deity, to run the experiment with no deity present and compare it to the same experimental run with the deity present. There’s the rub. No one has been able to do that so hypotheses assuming a deity are considered outside the realm of science and therefore untestable. More importantly these hypotheses cannot be evaluated on the merits of scientific theories because, well, they aren’t falsifiable.
This brings us right back to the “just a theory” statement. If we’re going to strive for clarity in our communication, we really should be working from the same set of definitions. So the next time someone tells you “it’s just a theory” ask them tenderly, politely, what they mean by theory in the first place rather than trying to prove or disprove your basic idea. You might head off an argument simply by agreeing to disagree on the meaning of the term. But be prepared to defend your definition too.
Enjoy the holidays!