Occam's Razor

Pick the explaination that causes minimal change to your belief system. An ego preservation heuristic.

I like to consider it a tool for ordering theories, before testing them (in hard science, of course).
It is not a tool that finds truth. It does not concern itself with truth values, it is a heuristic.

Let us suppose that, while fixing a snack for my good friends hamster bong and hamstergirl, I hear a rustling noise that comes from the living room. My question is: What made that noise ?.

1. My pet hamster has been freed by hamster bong and hamstergirl, and is currently frolicking in the wastepaper basket.
2. A burglar has entered the house.
3. Martians have landed in my backyard and they are paying a visit.
4. The ghost of Microsoft is rattling a paperclip chain.

Notice the trick: I have ordered some of the (infinite) possible theories in inverse order of assumptions number and strength necessary. In detail:

1. Hamsters are frolicsome.
2. Someone broke and entered.
3. Living beings exist on other planets.
4. Corporate entities return as ghosts and suffer for their sins.

The scientist in me (and in you, and in a lot of other rational people) will start with the first theory, and shout "Put the little bastard back in the cage, will ya ?". That tests the theory. If the test fails, then I will perform more tests. Only ultimately I will have to pull out the Martian-Zap-o-Mat, or the Blessed Linux CD in order to banish the noisy uninvited guests.

Really, Occam's Razor is not anything strange: what is strange is how people occasionally stop applying it.

The canonical example is the theory of ether. In the 19th century, physicists explained the transmission of light, and electromagnetic waves in general, by assuming the existence of an invisible substance filling the universe; this substance was called the 'ether'. Modern physics does not need this concept: the razor has eliminated it.

This nicely illustrates the opening remark: the reason we speak of Occam's razor here is that a concept was eliminated, not that theory was simplified; the hypothesis of ether was in fact replaced by a more complicated theory.

This razor may seem a simple tool of the mind, but it stands for an important shift in attitude in Western intellectual life, that first developed in Ockham's own time, the Renaissance, and gained force, particularly in the Anglosaxon world, up to the present day.

In scholastic thought, wisdom consisted in understanding the words of earlier wise men; it was the task of the intellectual to understand them and apply them in the real world. Words are put first.

In the newer scientific mode of thought, observed reality is put first: words, concepts, theories, are only tools to describe reality, and like all tools, they can be replaced by better ones. Wise words and concepts from the past are simply discarded when better explanations of reality are found that do not rely on them.

To a scholastic thinker, truth is found in books; what we have to do is understand them and see how they apply to our lives. The Bible writes: in the beginning was the word. Plato writes: reality is formed by the imperfect manifestations of absolute concepts. Descartes says: I can build up knowledge by pure reasoning, by carefully defining what my terms mean and exploring the logical consequences.

To a scientific thinker, the truth is out there: to be precise, we do not really have truth, since every truth is stated in terms of concepts, and concepts may turn out to be obsolete; the best we can hope for is certainty. This view is represented in the history of philosophy by Hume and other philosophers of the British school of empiricism.

The present-day battles between religion and science are really manifestations of the battle between scholastic and scientific thought:

In religion, the word is a given. The stories, explanations, concepts and formulations provided by the Holy Scripture are unquestionable: we can annotate them and interpret them, but to consider if they could be replaced with different, more accurate words, immediately takes us out of the realm of (the Christian) religion.

In science, on the other hand, the word is just a working tool. Darwin makes an honest attempt to explain how evolution works based on what he observed, but his words may be replaced with better explanations, should they come along, since what really matters is not the words, but the actual world and how it works.

This explains why the creationism debate is so thoroughly unproductive; the collision is not about matters of fact or explanations of facts, it is a collision between the scholastic and scientific intellectual attitudes. From a scholastic viewpoint, Occam's razor, or for instance, the notion of a scientific theory held in science, are utterly nonsensical. God-given concepts on the other hand - starting with God - are impossible to understand from a scientist's point of view.

Thanks to Purvis for remarks.

Citing Occam's Razor as "If two theories explain the facts equally well then the simpler theory is to be preferred," is both misleading and gratuitously incorrect. That explanation is, reflexively, the simplest interpretation, but neither necessary nor true. Let me explain, beginning from the beginning.

Depending on your source, the original Latin is one of:

• "Pluralitas non est ponenda sine neccesitate."
• "Frustra fit per plura quod potest fieri per pauciora."
• "Entia non sunt multiplicanda praeter necessitatem."

All meaning, roughly "Entities should not be multiplied without cause." Unfortunately, this is a little opaque for the English-speaking crowd, who tend to prefer:

"We are to admit no more causes of natural things than such as are both true and necessary."

This is a very important distinction. First of all, the word "simple," as used in the common misinterpretation, is highly ambiguous. Many people interpret "simple" to mean "obvious" or "straightforward" while others take it to mean "easy to understand" or "unburdened by rigorous proof." In all of these cases, wildly different conclusions can be drawn, and the use of the this, faulty, Occam's Razor is almost totally useless. This leads to many circular debates, especially about religion, and cosmology in particular, with both sides gesticulating madly and invoking Occam's Razor to no effect.

The correct interpretation, however, does not share these faults. It is also not nearly as powerful as its faithful abusers would like it to be. An attempt to explain it is: Given an understanding of natural phenomenon, it is incorrect to conclude that which is not required by the nature of that phenomenon.

A correct use of Occam's Razor would be a counter to the following argument: "There is thunder and lightening. Therefore, we know that Thor is upset." This is an invalid argument because a thorough understanding of thunder and lightening does not require that Thor is the cause. Note that the use of Occam's Razor in this case does not refute that Thor is the cause, it just denies that assumption based on the evidence. The distinction between denying the assumption and denying the fact is important, because, really, Thor could be causing thunder and lightening indirectly, and modern science just hasn't discovered this. Occam's Razor is useful because it admits multiple, indirect causes of events, not all of which are clear at any given time.

The flaw of Occam's Razor is that it presupposes a correct understanding of the natural phenomenon. Which reduces its usefulness greatly in discussions of cosmology, although it may still be used to counter leaps of logic.

In summary: It's a razor. Don't play with it unless you know how to use it.

Ockham's razor is named after William from the village of Ockham who propounded it. However it is often written Occam. It is not at all clear which is correct, in those days spelling was not nearly so standardised.

The reason that Occam's razor is needed is that it is always possible to come up with ever more bizarre theories to explain any behaviour of the world.

Is the moon pushed around by the beat of invisible, intangible faeries wings? Doubtful, but it may be difficult or even impossible to disprove. Or is it this newfangled gravity thing? Or does the earth create a suction? How can we prove it? We often can't.

But we need somewhere to stand to compare things against. Otherwise all of science become strictly a matter of opinion. That's what The Razor provides us.

In fact, Ockham's razor is way of building what statisticians call a "normal hypothesis"- the default theory that any new theory must be experimentally compared against in order to prove its mettle.

Contrary to popular belief Scientists or anyone else are welcome to believe whatever they want, but people in science will generally expect you to compare your theory experimentally against a reasonable Normal Hypothesis (see Standard Model); and try to disprove the normal hypothesis.

Interestingly, Ockham's razor may be applied to religion, and it generally states that there is no God. However, that does not disprove the existence of God, only that God cannot be the default scientific theory without direct evidence (God is a very complex concept, omnipotent, omniscient etc.); if you're religious, that's what faith is for.

Ockhams razor, or some similar concept, is both subtle but vital to science. It is practically the definition of Science.

Algorithmic information theory (AIT) provides a neat way of approaching Occam's Razor. If we have some result or set of results to be explained we can identify theories to explain them with programmes (or algorithms) which output that result (in some encoded form). Now Occam's Razor can be interpreted as stating that the "best" theory is the one with the shortest programme. In other words, this is a formulation of theoretical science as the compression of the information nature provides.

It is a remarkable fact that much of nature is highly compressible, or at least that approximate versions of nature are - for example, the entire of rigid body mechanics at a human scale can be expressed in Newton's three short laws, and as I understand it basically the entire of quantum mechanics is contained in one little partial differential equation - Schrodinger's equation. Of course, we can't actually know whether any theory we have actually is a compression of nature as it really is, or is just a compression of the approximation of nature we can measure. So it is quite conceivable that nature is in fact completely incompressible - random, in the AIT sense - in which case Occam's Razor could not be applied without shaving off some explanatory power, and the best theory would simply be "what is is what is".