I wonder if anyone has explored a quantitative measurement of the soundness of an argument. By soundness, I mean the extent to which the argument's premises are true and valid. By quantitative measure, I mean has anyone constructed a number that, if you glanced at it, gave you a sense of how sound an argument is?
2I'm sorry, but I don't know what you're asking for. What "quantitative measures"? Some examples or context, perhaps?– DennisAug 8, 2013 at 19:07
No, because there isn't even a soundness property of statements to be measured. Soundness is a property of deductive systems, not individual statements.– David HAug 8, 2013 at 20:12
@DavidH He might be thinking of something like a "sound argument", an argument that is valid with true premises.– DennisAug 8, 2013 at 21:11
3How is one supposed to quantify something without formalizing it? How do you know what to measure, or what even counts?– Rex KerrAug 8, 2013 at 21:36
@Dennis, yes, I am thinking of something like a "sound argument", as in one with valid and true premises.– Brash EquilibriumAug 8, 2013 at 23:10
An argument is sound if and only if it is both logically valid and all of its premises are true.
An argument is either logically valid or is not logically valid. There is no middle ground here. In terms of numbers, the logical validity of an argument can be either of the two values in the Boolean domain, which are
A premise is either true or is not true. In terms of numbers, the truth of a premise can be either of the two values in the Boolean domain, which are
Intuitively, the truth of a set of truth-bearers is a ratio of the subset of true truth-bearers to the superset. Basically, what I'm saying is that If half of an argument's premises are true then the set of that argument's premises is half true.
0is false and
1is true, then "half-true" seems like it would be
- If both are true, then the the value is
- If neither are true, then the value is
If we know that an argument has any invalid deductions, then we know that the argument is not sound.
- If we know that an argument has any false premise, then we know that the argument is not sound.
- So, for the soundness of an argument to be represented by a number that is not in the Boolean domain, then that argument would need to be missing some information. Arguments are made valid by reasoning with valid deductive steps and premises are made true by qualities of their extension in the real world. Since knowledge of valid deduction is necessary for interpretation, we can make determine that if there is missing information it is probably about the truth of the premises. This is not absolutely necessary, but we can pretend it is for this discussion because cases where we can't determine how to think logically are significantly less common than cases where we don't can't determine if a statement is true or not.
So, any "quantification of soundness" is going to require that we don't know the truth value of at least one of the premises. Now let's revisit your question: "Are there quantitative measures for the soundness of an argument?" Since we can determine this depends on the truth of the premises with unknown truth values, we can rephrase the question as "Are there any quantitative measures for the truth of premises with unknown truth values?"
When phrased this way it becomes clear that what you're actually asking for is Probability, defined on Wikipedia as "the measure or estimation of how likely it is that something will happen or that a statement is true". Probabilities are also ranged over the Boolean domain, but include real numbers and not just integers. So,
0 means that the statement will never be true and
1 means that the statement will always be true. Mapping probabilities to logical/set operations is pretty intuitive. Intersection of premises multiplies the probabilities of their truths to get the probability of truth of the composite judgment. So, if there is a 0.5 probability that P is true and 0.4 probability that Q is true then there is a 0.2 probability that both P and Q will be true. This type of math works equivalently for greater numbers of premises. That should be enough for you to get started.
The point is that the quantitative measure of the soundness of an argument can be reduced to the probability that all its premises are true.
Thanks for your excellent answer. This is very similar to the one I've come to, but you clarified several points. One thing that I find interesting is that, in the limit of infinitely many premises, and assuming that there is incomplete certainty in the truth of each premise, the joint probability of the composite judgment goes to zero. It's a built-in Occam's razor. Aug 10, 2013 at 9:29
However! I disagree with you about the measurement of validity. I believe it should be measured as the density of fallacies per premise, because we should prefer arguments that make fewer fallacies to ones that make more, and in informal argumentation, almost everyone will commit at least one fallacy. If you are interested in comparison, better to go with something that is likely to have more heterogeneity. Aug 10, 2013 at 9:31
@BrashEquilibrium something about your terminology is off. Premises don't have fallacies, because fallacies apply to logical derivations not truth-bearers. What exactly are you trying to get at with 'density of fallacies per premise'? Aug 10, 2013 at 17:01
@BrashEquilibrium i do think your observation about how this relates to Occam's razor is interesting Aug 10, 2013 at 17:05
again you are correct. There is no one-to-one or even one-to-many correspondence between premise and fallacy, although many premises can together represent one fallacy, and one premise on its own can be a fallacy. The reason why I suggest a "fallacy per premise" measure is because a more complex argument with more premises can contain more fallacies simply because it has more premises, and not because it is more fallacious than a simpler argument with fewer premises, but many fallacies per premise. Aug 11, 2013 at 4:58
If you like reductionism and you have an Occam prior, the answer is yes.
Given any phenomenon, the simplest explanation is best. But 'Simple' is a word in English, so what does that actually mean? Preferably in numbers, so you can compare which is simpler with a single complexity(argumentX)>complexity(argumentY) therefore Y?
Occam's Razor is often phrased as "The simplest explanation that fits the facts." Robert Heinlein replied that the simplest explanation is "The lady down the street is a witch; she did it."
There are two expansions on that which are isomorph to each other: Solomonoff Induction and Minimum Message Length.
Minimum Message Length:
The lady down the street is a witch, she did it. She did what, exactly? The shortest message (measured in bits of information) you can use to describe what happened is the Minimum Message Length. Shortest message wins. (Example: Explaining the theory of gravity and the starting state of a system is shorter than describing each position the system takes over time. Since the message is shorter with the Theory of Gravity, the Theory of Gravity is a simpler explanation that "it just happened like this."
The lady down the street is a witch, she did it. She is a what, exactly? Well she is [The simplest turing machine you can write that describes how she acts], and she did it.
It turns out that with some fancy math, it can be shown that explanation that takes the simplest turing machine to develop is also the explanation that has the shortest message length, so either works.
That is a QUALITATIVE measure of which argument is SIMPLER, which (in reductionism with an Occam prior) is also the argument that is sounder.
Yes, I'm familiar with MML, although I've not heard of Solomonoff Induction. Hrm. While I see this as potentially useful for comparing two arguments of equal soundness (i.e., equal probability of being true given the structure of its premises, and equal density of logical fallacies per premise), I don't think MML alone is enough to measure soundness per se. That said, I upvoted you because I should indeed incorporate Occam's razor into any metric I devise. Thank you. Aug 9, 2013 at 15:53
I just realized I don't have at least 15 rep on this Stack Exchange site, so I am unable to upvote you as I wished. :-( Aug 9, 2013 at 16:10
I will take your upvote in the spirit in which it was intended. Aug 12, 2013 at 6:56
There's your upvote now that I have the power to give them! Aug 14, 2013 at 0:53