Good books on Parsimony/Occam's Razor/Simplicity?

[luvluv]

Looking for in-depth treatments on the uses and abuses of the principle of parsimony in theory choice in science, and the applicability of the principle to disputed issues in metaphysics. Is there such a thing?

[comiezapr]

The principle of parsimony is really only used in specific arguments against a certian theory, or when two theories are contrasted. For instance a theory will be espoused, and then within it certian entities are found to do no theoretical work and so are unecessary. Or, and what is more frequent, one theory is reduced to another and there is shown to be a complete isomorphism between the theories in respect to ontology (what they posit as entities) exept for some particular entity in one of the theories. Consequently, the theory with the extra entity is discarded.

More frequent cases are when there are two theories that posit entities of completly different kinds. You cant use a principle of parsimony between these theories because they are qualitativly different; parsimony doesnt make sense.

As a result of this dynamic, parsimony is generally not used to motivate a particular theory. People dont come out and say "look how few entities i use" as an argument of very much weight. It has some wieght, but i would say almost none. I cant think of any instances where the principle of parsimony has been a deciding factor between the acceptance of two thoeries, either generally by a group of scientists/philosophers, or by myself. Let the entities roll in, provided they do some theoretical work. More entities usually means a better analysis anyway!

There is a principle related to the principle of parsimony that holds alot of weight; how close the entities a theory posits are to entities that are held to exist acording to common sense. This is the principle of common sense, and it is an extremely important principle to abide by, especially within philosophy.

I cant think of any work that deals with the principle of parsimony. Its part of an argumentative technique and doesnt really require much exposition to explain how it works. The arguments that it is used in are far to varied to taxonomize to put into a work, also.

[luvluv]

Well, I'm just thinking of it in terms of Swinburne's argument as opposed to, say, Max Tegmark's theory. If you're not familiar, Tegmark's theory is GUT proposal that suggests that all mathematically possible universes exist, and Swinburnes argument for the existence of God assumes that God, who is (on his account) the most simple personal agent possible, exists. Both accounts claim that the main advantage their systems have is parsimony.

But I wonder how one can make sense of either claim, since one claims parsimony for a system that has an infinite number of universes, whereas the other claims parsimony for a system that contains a being of infinite power. So, I'm wondering how to judge the dueling claims of parsimony? Which account of parsimony is most applicable here? Qualititative? Quantitative? Kalmagorov (sp?), the parsimony which favors the least number of variables in an equation describing the system? Swinburnes "brute fact" parsimony, which favors the system with the fewest number of unexplained "brute facts"? Is the principle of parsimony really applicable at all outside of science?

I'm looking for a book that addresses such issues. If there isn't one, there should be. Everybody and their mother seems to think that their system is either the most parsimonious or equally parsimonious to all other theories. Somebody should write a book analyzing these dueling claims of possession of the razor.

[comiezapr]

I certianly am not one of those people claiming to have a theory of maximum parsimony. The simplest theory is "stuff happens." This is completly uninformative exactly because it is parsimonious.

Parsimony doesnt play a role inside or outside science, and philosophers that claim thier theory is best because of "parsimony" are rather mistaken; parsimony means nothing except in the isolated circumstances i mentioned.

As to the particular theories in question, i have no aquiantance. Arguments about the fundamental nature of why the world is the way it is are generally ... lame. They dont really explain anything. Saying "it is because it is" is the best thing to do. If theres some theory behind the way things are then an obvious question is where the entities for that theory come from. The answer would be, again "it is because it is." There is no need for an explanation because the explanation does do anything. There is no redescription of facts since the facts in question simply arent there. There is no elucidation of how things are because all of the theories (including "it is because it is") are completly compatible with everything we know about the world. The explanations yield no insight into anything at all.

The explanation doesnt conform to the principle of parsimony because it posits more entities than the "it is because it is theory"; the so-called explanation needs all of the entities in the "it is because it is theory" and more, clearly a violation of the principle of parsimony in its isolated form i gave it a post ago. Any theory that is explaining why the world is the way it is (and this is what i take these theorists to be doing) are completly mistaken. Theyre acomplishing nothing that can be defended scientifically or philosophically; theyre retarded.

Whenever there is a case where two people claim to be conforming to the principle of parismony, usually niether are right. Parsimony applies to numerically more entities, or brute facts. When there is no 1-1 correspondance of entities or brute facts between theories the principle of parsimony simply makes no sense. Its like asking if theres more water or people in the world; it makes no sense without a correspondance between the two domains. What they want to claim is that their theories are more "natural" or "simpler" but these are vague things that have almost nothing to do with parsimony. (Except of course that there shouldnt be entities that do no theoretical work.)

Simplicity is to vague a concept to write a book on; simplicity is supposed to be an intuitive thing that you should know about before theorizing. Simplicity has close affinities to arguments from common sense; theories that are more commonsensical are better theories.

[Hugh Nose]

Simplicity by Elliot Sober, Oxford University Press (originally)

Hugh Nose

[mirage]

Hehe! I should be able to give you a weighty reference, but I can't.

The principle of parsimony is really only used in specific arguments against a certian theory, or when two theories are contrasted. For instance a theory will be espoused, and then within it certian entities are found to do no theoretical work and so are unecessary. Or, and what is more frequent, one theory is reduced to another and there is shown to be a complete isomorphism between the theories in respect to ontology (what they posit as entities) exept for some particular entity in one of the theories. Consequently, the theory with the extra entity is discarded.Very well stated.

More frequent cases are when there are two theories that posit entities of completly different kinds. You cant use a principle of parsimony between these theories because they are qualitativly different; parsimony doesnt make sense.It is certainly difficult.

As a result of this dynamic, parsimony is generally not used to motivate a particular theory. People dont come out and say "look how few entities i use" as an argument of very much weight. It has some wieght, but i would say almost none. I cant think of any instances where the principle of parsimony has been a deciding factor between the acceptance of two thoeries, either generally by a group of scientists/philosophers, or by myself.
Special relativity.

All the predictions can be reproduced by a modified aether theory, which keeps an absolute reference and has lengths and timekeeping modified by velocity relative to it, according to the same Lorenz transforms.

It was rejected as less elegant. I.e. it has a rest reference frame that does no theoretical work.

Let the entities roll in, provided they do some theoretical work. More entities usually means a better analysis anyway!Well that is the whole point isn't it? If they don't change predictions to better fit observations, they aren't doing this work.

There is a principle related to the principle of parsimony that holds alot of weight; how close the entities a theory posits are to entities that are held to exist acording to common sense. This is the principle of common sense, and it is an extremely important principle to abide by, especially within philosophy.Yes, that's a really great principle. It really helped with special relativity, and it served us really well when we believed in wood spririts and witches and things. It's still doing great service today to support benevolent all powerful beings of no observational consequence.

I cant think of any work that deals with the principle of parsimony. Its part of an argumentative technique and doesnt really require much exposition to explain how it works. The arguments that it is used in are far to varied to taxonomize to put into a work, also.
Most of the work is of a mathematical bent. Optimising Model fit to data, artificial intelligence, information theory.

[mirage]

I certianly am not one of those people claiming to have a theory of maximum parsimony. The simplest theory is "stuff happens." This is completly uninformative exactly because it is parsimonious.

Which is why the best criteria for judging parsimony include both the theory and the extra data needed to reproduce the actual observations to a given accuracy. Unfortunately this only really works in a simple formal system, but here you can combine both the theory and the necessary free variables to reproduce the observations in a single measure, which produces a trade off between a simple model at the expense of lots of extra data, and a complicated model with less need for other data.

As to the particular theories in question, i have no aquiantance. Arguments about the fundamental nature of why the world is the way it is are generally ... lame. They dont really explain anything. Saying "it is because it is" is the best thing to do. If theres some theory behind the way things are then an obvious question is where the entities for that theory come from. The answer would be, again "it is because it is." There is no need for an explanation because the explanation does do anything. There is no redescription of facts since the facts in question simply arent there. There is no elucidation of how things are because all of the theories (including "it is because it is") are completly compatible with everything we know about the world. The explanations yield no insight into anything at all.I tend to agree.

The problem with the theories you mention, luvluv is that we can't reduce them to easily comparible terms. For example "simple being" and "God" are terms far too woolly to have any real meaning, let alone a well defined parsimony cost.

Of course, if you are talking about a system with the intent to realise everything that we see realised in the world, then it must have at least the complexity of that world, and so the two are rather easily compared. "Simple" is a transparent attempt to avoid this clear cast iron conclusion.

Whenever there is a case where two people claim to be conforming to the principle of parismony, usually niether are right. Parsimony applies to numerically more entities, or brute facts.These are both crude guides that are designed to approximate the algorithmic complexity of a description. IMHO, of course.

[comiezapr]

All the predictions can be reproduced by a modified aether theory, which keeps an absolute reference and has lengths and timekeeping modified by velocity relative to it, according to the same Lorenz transforms.

No such aether theory was ever espoused that was empirically adequate. Relatvity also has various implications that an ether theory wouldnt have, which is why it is accepted.

The initial acceptance of relativity was based on, presumably, the thought experiments Einstein went through and offered to the physics community. It was, in short, accepted because, after reflecting on the nature of physical systems generally, the theory seemed to be commonsensical.

Im not sure exactly what kind of parsimony relativity affords over an ether theory anyway. Do contractions count as part of the count? What about
the various frames of refrance (in which case there would be an infinite amount in both theories!)? What does the principle of parsimony act upon in this situation? My conclusion is obvious, nothing; the principle of parsimony is junk except in isolated situations.

Which is why the best criteria for judging parsimony include both the theory and the extra data needed to reproduce the actual observations to a given accuracy. Unfortunately this only really works in a simple formal system, but here you can combine both the theory and the necessary free variables to reproduce the observations in a single measure, which produces a trade off between a simple model at the expense of lots of extra data, and a complicated model with less need for other data.

Im not sure what this is saying. Theories should be judged both by how parsimonious they are and how well they fit data? Clearly this isnt the case because "stuff happens" is still the best theory. It fits all the data and couldnt be simpler!

The problem isnt with fitting data, the problem is in discriminating between various differences; a theory with more things in it can make finer discriminations. Parsimony is something to AVOID, as a general rule, because of this phenomena; the more stuff a theory has the better it can represent various things. Parsimony is only good as an argumentative tool in the very specific circumstances that i outlined.

There is another thing you could be saying: a theory that is created to EXPLAIN some facts should be parsimonious in the mechanism used to explain the facts. I doubt that this is correct. Theories should conform to common sense when explaining things, as best as possible; they should not conform to parsimony. It could happen that common sense and parsimony coincide, but this is highly doubtful in the general and hard to find in any particular cases.

I dont really know what this paragraph is saying, to be honest. Its just alot of stuff run together it seems.

Most of the work is of a mathematical bent. Optimising Model fit to data, artificial intelligence, information theory.

Where is parsimony used as part of an argument in artificial intelligence or information theory? I havent ever seen an argument like this in these areas. I dont even think that an argument of this sort could be used in these areas! What is optimising model fit to data and how is this an endevour with a "mathematical bent?" I just dont know where this comment came from, there needs to be specific examples.

Two things should be gotten out of this post. The first is that the general "parsimony" is really vague (since it isnt clear what numerically distinct things the principle of parsimony is to act upon). The second is that, as a general rule, parsimony reduces us to rediculous theories of the variety "stuff happens," and so should be explicitly avoided; adherance to common sense is the principle to follow and is probably what the principle of parsimony trys to capture in a (deceptivly more definite) phrasing. Either the principle of parsimony is a paraphrase of adherance to common sense, or it is a highly definite rule that applies to numerically distinct things, in which case it is rediculous as a general rule. There is no middle ground here.

Edit: Ive gotten considerably off topic and im sorry.

[mirage]

No such aether theory was ever espoused that was empirically adequate. Relatvity also has various implications that an ether theory wouldnt have, which is why it is accepted.Incorrect, as I have already said. An aether, i.e. absolute rest frame theory can be made consistent with all the observational predictions of SR. If you like I will find out the name of it, but it should be fairly obvious that it can be done. Instead of having a constant c relative to any inertial frame, you Lorentz tranform the measuring equipment so that it only appears to be constant.

The initial acceptance of relativity was based on, presumably, the thought experiments Einstein went through and offered to the physics community. It was, in short, accepted because, after reflecting on the nature of physical systems generally, the theory seemed to be commonsensical.Because we naturally apply parsimony and it is "common sense" that if you can reproduce the same observations by treating measurements at face value and discarding an absolute rest frame, then you probably should do so.

Im not sure exactly what kind of parsimony relativity affords over an ether theory anyway.I have already made this quite clear. It lacks an absolute rest frame (not to mention velocity dependent transforms in length and time) which together are of no observational consequence.

Velocity of an inertial frame relative to this absolute rest frame would be an extra piece of information of no observational consequence.

Do contractions count as part of the count? What about
the various frames of refrance (in which case there would be an infinite amount in both theories!)? What does the principle of parsimony act upon in this situation?The complexity of the overal description. The number of reference frames is entirely and clearly irrelevant, just as the number of stars was irrelevant to the parsimonious conclusion that the sun is also a star (i.e. you reduce two entities to one by identification.)

SR does not need to list every possible reference frame relative to every observer! It just defines what a reference frame is. Once. And it makes no ontological commitments. It need be no more than a proceedure that defines which inertial observer you are talking about.

This is in contrast to having an absolute rest frame. This must exist as some kind of object in the modified aether theory, since velocity relative to it has real physical (but not observational) consequences in that theory. Changes in length, that kind of thing.


My conclusion is obvious, nothing; the principle of parsimony is junk except in isolated situations.That conclusion is based on an imperfect understanding. Furthermore, I'm not sure what you mean by "isolated". It seems to be that you are saying that parsimony is either obvious or ill defined.

It can be ill defined but it has not always been obvious. Lots of people believe in God, for example. Others struggled with the idea that the sun is just another star, although it is clearly the more parsimonious conclusion.

Im not sure what this is saying. Theories should be judged both by how parsimonious they are and how well they fit data? Clearly this isnt the case because "stuff happens" is still the best theory. It fits all the data and couldnt be simpler!It reproduces none of the data. It does not make accurate predictions. I.e. it does not fit the data well.

"Stuff happens" gives precisely no information about the world.

It would require an enormous amount of extra information to do this, which would all be unstructured. None of the relational information present in the observations is contained in "stuff happens".

The standard model of physics requires less extra information to reproduce observations, since it encodes information about the relation between observations in its theory.

In information theory and artificial intelligence, optimising the efficiency of this modelling of a set of data results in good predictive performance.

The problem isnt with fitting data, the problem is in discriminating between various differences; a theory with more things in it can make finer discriminations.You are talking about data fit. Data fitting does not mean "compatible with". It means fitting well. A suit the size of an appartment building does not fit you well.

Parsimony is something to AVOID, as a general rule, because of this phenomena; the more stuff a theory has the better it can represent various things. Parsimony is only good as an argumentative tool in the very specific circumstances that i outlined. No, I'm afraid your conceptions of the terms of this discussion bear no relation to the context I have used them in. There is a lot of work on predictive modelling and parsimony in the context of machine learning and none of it suggests it should be AVOIDed.

There is another thing you could be saying: a theory that is created to EXPLAIN some facts should be parsimonious in the mechanism used to explain the facts. I doubt that this is correct. Theories should conform to common sense when explaining things, as best as possible; they should not conform to parsimony. It could happen that common sense and parsimony coincide, but this is highly doubtful in the general and hard to find in any particular cases.Yes, what a brilliant idea. Clearly SR is just "common sense". Clearly when the most parsimonious hypotheses conflicted with the contemporary "common sense" that the earth was flat, that the earth was the center of the universe, that the solar system was unique in the universe, that the sun was different to the stars, that there was a single absolute rest frame and that time and distance were well defined regardless of the motion of the observer, that life was designed by a Creator, clearly in all these cases common sense was the best guide.


Where is parsimony used as part of an argument in artificial intelligence or information theory? I havent ever seen an argument like this in these areas.Why do you think you should have? You aren't even aware of what data fitting refers to.

I dont even think that an argument of this sort could be used in these areas! What is optimising model fit to data and how is this an endevour with a "mathematical bent?" I just dont know where this comment came from, there needs to be specific examples.
http://www.inference.phy.cam.ac.uk/mackay/itprnn/book.html

You could read that book for an introduction to information theory. It mentions Occam's razor and occam factors several times.

This is a general page on the razor and inductive inference
http://www.csse.monash.edu.au/~dld/Occam.html

and this give a very brief idea of what I was on about regarding model fitting and minimising total information (i.e. minimum message length, MML)
http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/DAVIES1/rd_mdl.html

Two things should be gotten out of this post. The first is that the general "parsimony" is really vague (since it isnt clear what numerically distinct things the principle of parsimony is to act upon).This is true. That doesn't mean that it can't be well defined in principle.

The second is that, as a general rule, parsimony reduces us to rediculous theories of the variety "stuff happens,"That is quite wrong, and demonstrates a basic misunderstanding of the principle.

Even in the most unsophisticated formulation, parsimony is about the simplest hypothesis that fits the data. You don't seem to have grasped the sense of "fit" in this context. It means sucessfully predicts or retrodicts observations.

P.s. it's "rIdiculous". I know you don't like spelling corrections, but you are slightly grating to read with these repeated obvious errors that don't even work phonetically. You may choose to continue to make them, of course, but alternatively you could use a spell checker as many people on here do as a courtesy to any readers.