# Can every idea, including mathematical ideas, be reduced to a series of simpler ideas, without information loss?

You would naturally think this is the case, since most ideas could be explained using a greater number of simpler words; but there are certain concepts in mathematics that are difficult to explain even when using a lot of simple words, especially in topology. So I was wondering if any philosophers ever pondered this, and if there's an irreducibility of some concepts, that makes it impossible to break them down without information loss.

• There are in the end the simplest words, which cannot be explained in terms of simpler ones because there aren't any simpler ones. Commented Apr 2 at 3:05
• i.e. if you have an ordering of words by simplicity, there are a finite number of words in the English language so there have to be some words of maximum simplicity in your ordering. These words cannot be explained in terms of any words of greater simplicity. Commented Apr 2 at 3:10
• you've just stumbled upon kolmogorov/chaitin randomness! plato.stanford.edu/entries/chance-randomness/#KolmCompRand
– ac15
Commented Apr 2 at 3:32
• This is not a full answer to your question, but you may be interested in Guy Steele's presentation Growing a Language. Its an hour long, and I recommend reading the transcript along side it, but he went down a thought process very similar to yours. He did it for computer science, not mathematics, but I think his conclusions (not to mention his most excellent demonstration of the idea) will apply for you. Commented Apr 2 at 15:15
• His question is not about Kolmogorov complexity, he is talking about simple words not short sequences of words. He explicitly says the sequences of simple words would be longer than the idea itself. So it is not about finding a minimum description length. Commented Apr 2 at 17:04

There's a quote in Pascal's Pensees (#20-21) which seems closely related:

(#20) Order.—Why should I undertake to divide my virtues into four rather than into six? Why should I rather establish virtue in four, in two, in one? Why into Abstine et sustine[10] rather than into "Follow Nature,"[11] or, "Conduct your private affairs without injustice," as Plato,[12] or anything else? But there, you will say, everything is contained in one word. Yes, but it is useless without explanation, and when we come to explain it, as soon as we unfold this maxim which contains all the rest, they emerge in that first confusion which you desired to avoid. So, when they are all included in one, they are hidden and useless, as in a chest, and never appear save in their natural confusion. Nature has established them all without including one in the other.

(#21) Nature has made all her truths independent of one another. Our art makes one dependent on the other. But this is not natural. Each keeps its own place.

However, while Pascal is talking about the inability to unify closely related concepts using a simpler general principle, your question is more about the ability to break a concept into multiple simpler ones. But it seems at least an indirect implication of Pascal's opinion here that there is a sort of "rock bottom" of ideas which cannot be simplified further, thus relating to some extent with your question.

Can every idea including mathematical ideas be reduced to a series of simpler idea without information loss?

[Is] there an irreductibility of some concepts that makes it impossible to break them down without information loss.

The way these questions are formulated makes it hard for me to understand what the OP actually wants to ask. But let me try.

### Attempt 1

"Reduced to simpler ideas" here - as made clear by context - is meant to mean "explained using a greater number of simpler words". But the OP apparently believes that certain terms may still remain difficult, even when a lot of simpler words are used. So, the question seems to be asking

Do we at some point, perhaps after multiple rounds of simplification, reach a stage where we just cannot simplify an "idea" further, but yet it remains "difficult"?

We can think of a scenario in which a math teacher is trying to explain some idea (for instance the idea of a vector in linear algebra) to a student, and where both the student still doesn't get it and the teacher cannot think of another way to simplify the idea. But that's not a philosophical problem. Either the student or the teacher of both may simply not be smart enough to understand or explain-in-simpler-terms the particular idea or the student may not have the required background knowledge and understanding. (I would be able to explain "infinite" to a 7-yr old "You can just always make bigger and bigger numbers and you never ever get to the end! See, we can even make a game of it. 'Cause numbers are really games - did you know? If you can give me the highest number in the world, you win... but you can never win... wanna try?!" And I could probably explain that the sky above us is not "infinitely high" in this sense. But I would not be able to explain Cantors' diagonal argument to a 7-yr old.)

So, is there a philosophical question? Perhaps like this

Do we (any intelligent creature sufficiently like us) have any totally simple ideas? I.e. ideas that are not composed (somehow) of other ideas.

It's not totally clear what that might mean. It's a question similar to the question whether or not there are "elementary facts" (as posited in Wittgenstein's Tractatus for instance) and/or whether we need to use "elementary fact" as a kind of ideal when speaking about facts. You could also think of (for instance) Euclid's Elements. For instance, take the definitions of 'point' or 'line' at the start of the definitions in Book 1:

A point is that of which there is no part.

And a line is a length without breadth.

A point is the simplest possible object in Euclidean geometry. But is it a concept that cannot be further simplified?

Apparently "point" is not a simplest concept, since it's defined in terms of "part (and whole)". So, part-and-whole is a simpler concept? Or should we say that there is no real difference in terms of simplicity: the geometric concept and the mereological one are on par as it were. Using a definition only posits some relation; it doesn't (need to) imply that one concept (or class of concepts) is somehow more simple (or more "fundamental")?

Other example. Take the idea of "natural number". This idea is an abstraction based on the act of counting. Can we explain this in simpler terms? What is the simplest idea or intuition involved? We don't learn what numbers are by being given verbal explanations, of course - we learn 'what' they are by learning to count. But doesn't this involve understandng or grasping a concept ('the concept of natural number')? (I believe we're getting into dangerously choppy philosophical waters here since we seem to have a strong tendency to reïfy patterns of activity and assume there must be corresponding 'concepts' or 'mathematical objects' in some conceptual realm - with an unclear relation to for instance the psychological realm.) In some sense, surely, learning to count is based on a cognitive skill and on 'grasping a concept'. But if so, what is that concept? If there is one - we might assume (with Brouwer) that it must be the naked intuition of time, the intuition of two-in-one: "one-thing-and-then-another-thing". Is this the simplest idea possible?

Personally, I am sometimes tempted to think that that idea - the naked intuition of time, with all empirical clothes removed, is the most simple idea possible here. But somehow it's also the most difficult idea, since I have no idea what that is - it seems to point to some kind of core paradoxical experience that we just cannot completely describe in any language. Are we just dazzling ourselves with words here, creating a kind of illusionist show, or is there a real experience underneath this? If there is a real experience, why cannot we express this more clearly? (Or is this clear enough?)

### Attempt 2

Concepts do not exist in isolation. They are embedded in systems of other concepts -- signs, language games, forms of life. Grasping a new concept means that everything around it changes - or gets a new meaning.

The reductions that mathematicians may perform (for instance reducing number theory to set theory) are always based on equivalence relations. So, there can never be relevant "information loss", otherwise the reduction would be invalid. In sofar as the reduction is new to somebody, there will be information gain to that person. The purpose of those reductions is also never "simplification" - "simplification" is irrelevant. The purpose can be just to demonstrate that there is an equivalence or to show new, possibly more rigorous methods of proof. (If you've ever gone over an actual reduction of some NP-complete problem to another one, you know how complicated this can get. Even understanding a reduction may not be simple, let alone finding a new one. For a random, but truly beautiful example that I still don't completely understand, see Indyk and Backurs Edit Distance Cannot Be Computed in Strongly Subquadratic Time (unless SETH is false))

In the context of teaching/learning concepts where we speak of "trying to simplify" things for the student - "simplification" is something entirely different from what normally is called "reduction". We're not "reducing" anything to anything else. We're trying to push the student towards a different way of looking at things, mostly by using analogies that may be more fruitful than their current way of looking.

I vividly remember how as a 12-yr old I came to fully grap the concept of vector in a vector space. Since I was one year ahead of the other students, I always worked alone. Our linear algebra book was based on the New Math method that emphasized set theory (which I really liked), but it never even mentioned geometric intuitions - there was not the slighest trace of those (which I now regard as a horrible lack of educational wisdom). So, I had a lot trouble making sense of all those pairs of numbers and their abstract operations; the problems seemed pointless and I even had trouble remembering some methods since it was all pure, tedious numerical algebra. When I complained about this to my teacher, he just said: "Vectors are just arrows. Arrows in a space." That hit me like a lightning bolt. Suddenly everything I had been doing so laboriously fell into place.

But is the concept of an "arrow" simpler than the concept of a "number pair"? No. It's equivalent, but different - it hints at a completely different -- world-changing -- way of looking at the same world. As Wittgenstein wrote: Die Welt des Glücklichen ist eine andere als die des Unglücklichen. The world of a happy person is a different world as the one of the unhappy person. (Tractatus 6.43) I believe Wittgenstein meant this as a paradox. As a non-paradox it's a trivial statement. :)

## Attempt 3 - Jumps in the dark. Sudden revelations.

Can every mathematical idea be reduced to simpler ideas without information loss?

The question itself is a bit confused. It suggests, in context, that "reducing an idea" means "explaining an idea". Let's ignore the term "reduced and reformulate.

Can every mathematical idea be explained in simpler ideas without information loss?

This is still a bit confused, since it suggests that an explanation could "lose information". But, surely, when we present `a` as explanation of `b` then for all relevant purposes (in all relevant interpretations of "being as informative as" or "amount of information") they should either contain the same amount of information or `a` should contain more information. So, "without information loss" seems redundant. Let's simplify the question again:

Can every mathemathical idea be explained in simpler ideas?

This is still rather confused, since it suggests that certain "ideas" are inherently simpler than others. It suggests that somehow we can measure the complexity of "ideas".

In fact, we can (algorithmic information theory/complexity theory), if we interpret "ideas" as "representations in some language" but this seems not really relevant for the question. (But see @JD's answer for a somewhat different take on it.)

But what is meant by "simpler ideas" and "explaining" then? I see (at least) two very different ways of explaining:

(1) We can try to explain concepts in terms of "basic" definitions and axioms. The main question then is: Did we cover every aspect of the concept that is relevant?

This form of explanation does make everything simpler, ultimately, since it involves analysing and breaking down a concept into it's constituents. But the explanation itself can take quite a while to understand; it can also take quite a while for the community to come to agreement about it and to settle on a standard explanation. (Example: Peano axioms capturing the concept of natural number. Example: Very early confusions and controversies in the development of axiomatic set theory.) But this is (I assume) not the meaning of "explanation" that the OP had in mind.

(2) Explaining a concept or method to students who are still struggling with it.

In this case we're really never "looking for simpler ideas". That formulation seems misguided and leading us astray. (The word "idea" is like a freight train full of toxic waste constantly in danger of getting off the rails.)

The student (which may be me) needs to acquire a new concept, a new skill. The student needs to start to see new relationships - that is, start to see them without much extra effort. Instead of (merely) talking about "explaining in words" or "explaining in simpler ideas", it seems more fruitful to look at this as similar to (or really identical to) developing new practical skills. The student needs to find a way or a new way to "look at" a problem. So, a teacher may present analogies or demonstrative hints (drawing attention to this part of that part of a problem or method, refocussing). The student then still has to "grasp" the analogy and "do the same thing". (For a beautiful analysis of how incredibly rich and multi-faceted the concept of "doing the same thing" is, see Douglas Hofstadter's and Melanie Mitchell's CopyCat Project)

Sometimes, some concept may still remain difficult. I remember how in my freshman year I had a lot of trouble understanding the axiom of choice (and seeing it's point). Looking back, I realize that this was entirely due to the fact that initially I had studied no proofs in which it was actually used. To understand a mathematical 'concept', it's vitally important that you see it in use. And then also actually use it yourself (doing sums, solving standard puzzles, simpler puzzles). Understanding a concept implies you have used it - so you have used it togheter with other concepts - and are able to use it in some contexts. Understanding grows and deepens in sofar relationships (to other concepts, to other fields, other classes of concepts) are discovered (or established).

Sometimes, when a concept still remains difficult, what is needed is not "simpler words" -- since in that sense all efforts have been exhausted -- but temporarily abandoning the effort and diving into an apparently completely different field. This can make it possible to then later "suddenly" "see" the deep similarities and differences, the analogies, that one was not able to see earlier.

When Anrew Wiles proved Fermat's Last Theorem, he didn't merely add one more theorem to the set of all mathematical theorems; he made it possible to see more clearly the relations between different mathematical fields - fields that the community knew might be involved in finding a solution but that nobody had been able to actually completely bridge yet. Mathematics is like building bridges in total darkness. We know the endpoints, `a` and `b`, but have no idea how to jump. How to make that jump or how to make the bridge cannot really be expressed in words, I believe - at least not until the jump actually has been made successfully. As evidence of that, it seems very typical that when a mathematician (or a student) does 'see' the solution - the way forward - this appears as a sudden revelation:

I was sitting at my desk examining the Kolyvagin–Flach method. It wasn't that I believed I could make it work, but I thought that at least I could explain why it didn't work. Suddenly I had this incredible revelation. I realised that, the Kolyvagin–Flach method wasn't working, but it was all I needed to make my original Iwasawa theory work from three years earlier. So out of the ashes of Kolyvagin–Flach seemed to rise the true answer to the problem. It was so indescribably beautiful; it was so simple and so elegant. I couldn't understand how I'd missed it and I just stared at it in disbelief for twenty minutes. Then during the day I walked around the department, and I'd keep coming back to my desk looking to see if it was still there. It was still there. I couldn't contain myself, I was so excited. It was the most important moment of my working life. Nothing I ever do again will mean as much.

[Andrew Wiles' recollection of how he finally fixed the error in his original proof - quoted by Wikipedia]

It's a little ambiguous what it means to simplify ideas without information loss. Simplification might be understood on two levels, on the level of syntax and on the level of semantics. Of course, it's not clear from the question what information should be understood as, and Luciano Floridi and others have been conducting a philosophical analysis of the topic. An overview of their findings is proposed in the SEP's article Information. Alternative notions of information that are based on semantics has an entirely distinct entry in the SEP: Semantic Conceptions of Information. In the SEP, there are even analyses of information in regard to logic: Logic and Information.

Thus, simplification of syntax lends itself to the notion of data decompression of which Kolmogorov complexity is fundamental for describing how a representation can be simplified by returning it from a state condensed essentially bit-wise. This analysis of the encoding and entropy of information is information theory and is mathematical as earlier engineers and Claude Shannon approached the problem. Here, we are concerned with finding an equivalence between the smallest string to represent information at the tradeoff of making production more complicated. Decompression makes strings simpler algorithmically, but at the cost of increasing their lengths. Here the emphasis is not on the meaning of the text so much as the symbols used to represent the meaning.

Besides a bit-wise or character-wise analysis of a string, we might talk about simplifying the units of meaning of a text, a semantic simplification, though this is a little less clear. Fundamental to language is the principle of compositionality, which means that units of meaning, in natural languages called morphemes, can be composed in complicated arrangements. For instance, in English antidisestablishmentarianism has a number semantic units strung together: 'anti' means opposite or against, 'dis' means not, and so on. The question arises as whether an explanation of the term such as the definition is simpler (cognitively in this regard) or the term itself is simpler. And what of a semantic analysis of meaning stemming from a sentence containing the word?

In philosophy, one way a theory might be simplified is by parsimony of entities as in the language of Occam's Razor. Today, we talk about ontological primitives of a theory. If a scientific explanation and theory is information, then we can simply talk about which is more parsimonious along ontological and epistemological dimensions. Should this count as "simpler ideas without information loss"? Barry Smith in his article "Ontology" for the Blackwell Companion to Computing and Information tackles how modern applied ontologies affects the traditional notions of meaning and information. Consider the difference between a function that generates a string and a string itself. If these are interchangeable, should entities be seen as simpler than processes (he discusses substantalist and fluxist ontological positions)?

Another difficulty in answering your question is that it would be necessary to have a theory of ideas or concepts to make rigorous the notion of complexity of ideas or concepts. Here, the SEP tackles the notion of Concept. Are ideas the same as concepts? What gives an idea structure? Both notions (note the reach for the term 'notion' as a synonym for idea and concept) are often thought of as units of thought. How does one begin to simplify a unit of thought? Syntax is behavior, manifested by linguistic competence, but semantics is understood as mental activity, manifested by linguistic performance. How does one talk about simplifying performance distinct from competence?

Can every idea including mathematical ideas be reduced to a series of simpler idea without information loss?

So, the short answer is yes, at the syntactic level, any idea, or at least the expression of an idea in language, can be reduced to a simpler form without information loss. At the semantic level, it's not entirely clear what it means to "reduce mathematical ideas", but it can be said that complex mathematical formalizations certainly can be explained in a series of far simpler statements to aid comprehension.

• In algorithmic information theory, however, the compression of a string (a code) into a shorter string is not a "simplification"; it reduces the code to a more complex, more random code that can generate the input code (given a universal turing machine). So, that input code could then rather be seen as a "simplification" of the compressed, more random code. The compressed code is shorter, but not in any way "simpler" than the uncompressed code. Commented Jul 31 at 17:38
• Other, little footnote - In the last paragraph, you don't seem to be using the words "syntactic" and "semantic" in their usual senses. I agree with the last sentence, but it's not clear what you mean by the contrast here, or by "on the syntactic level" - Kolmogorov complexity doesn't involve any notion of syntax or semantics, afaik. Also, it's not true that any representation can be further reduced. Philosophically this would lead to an infinite regress. In complexity theory there is a limit to the compressibility of information. (Chaitin's number omega, ... Commented Jul 31 at 17:46
• @mudskipper See what you did there. I flipped the script to make the answer consistent with your claims, thanks. I was muddled on how production rules should function in the context of generating a string character-wise.
– J D
Commented Jul 31 at 17:58
• @mudskipper As far as syntax and semantics, I'm trying to use in a way that's consistent with the notions defined in physical computational theory. But to be honest with you, I still haven;t mastered the notions of the competing theories of Piccinini's 'mechanistic account' and Shagrir's 'semantic view of computation'. So, to be honest, I'm sort of swirling around with all sorts of different notions of syntax and semantics, trying to find a way to couch different theories in a physical account. For instance...
– J D
Commented Jul 31 at 18:05
• @mudskipper So, I'm still floundering on how best to use syntax and semantics in philosophical language. To be fair to, metasemantic theories are plentiful (consider the difference between truth-conditonal semantics and cognitive semantics), and none of my coursework was in complexity and computational theory... I just have my textbooks... so I tip my hat to you on a better command of CS.
– J D
Commented Jul 31 at 18:17

If you're talking about philosophy and not data compression, then no. You can expand meaning down to the atomic level of individual words with agreed-on meanings and canonical relations between those words.

Information is only lost when it is condensed into more general concepts.

• "You can expand meaning down to the atomic level of individual words with agreed-on meanings and canonical relations between those words." - and can you make it simpler than that?
– TKoL
Commented Jul 31 at 14:49
• No, because you're encoding information created by humans. It as a finite frequency -- Information per unit time or volume or whatever. Words mean only what we intend them to mean. There isn't anything below that. McLuhan said, "The meaning of meaning is meaning." The plumage don't enter into it. Commented Jul 31 at 15:11
• The ops question was, can EVERY idea be reduced - your answer says yes, but it seems you've found an example of an idea that can't be reduced - specifically an idea expressed at the atomic level of individual words, right?
– TKoL
Commented Jul 31 at 15:33
• @TKoL Yes, but the quantum of spoken information is the word. These objects are made by humans, and they only contain a finite and generally very small amount of information. You can't dig deeper 'cause there's nowhere to go. Why, I feel that way myself sometimes. Commented Jul 31 at 16:25
• @TKoL thank you [curtsies] Commented Jul 31 at 16:40

...and if there's an irreducibility of some concepts, that makes it impossible to break them down without information loss.

I'm not sure what you mean.

The atom is a concept that can't be broken down without information loss. Clearly, an atom can be reduced to components (electrons, neutrons, and protons) but information specific to an atom (physical and chemical properties) is lost.