I am trying to wrap my head around the principle of conservation of information as formulated by Leonard Susskind and others, which seems to me at first glance to be incompatible with the 2nd law of thermodynamics.

Per the principle of conservation of information, information is never lost, so what happens if we deleted information from a hard drive or burn a note so that nobody can ever read what was written on it?

Per Susskind, the information is now contained in the heat that is dissipated by the deletion or burning process. It is no longer accessible to anybody, but it somehow still exists as "hidden" information (See this post in the physics SE).

But how can it still be information if it can never be known by anyone?

Per the wikipedia definition of information:

As it regards data, the information's existence is not necessarily coupled to an observer (it exists beyond an event horizon, for example), while in the case of knowledge, the information requires a cognitive observer.

But I assume this means that even though there is no cognitive observer with access to the information, that it should be potentially accessible to a cognitive observer.

If on the other hand, the information has been made permanently inaccessible by the laws of physics (namely the 2nd law), how does it still qualify as information?

From an empiricist or positivist point of view, this no longer qualifies as information, by definition.

But even from a rationalist point of view, I fail to see how one can still consider this information?

Is there a way to define information from a rationalist point of view, such that this "hidden information" concept makes sense?

Or are the physicists just misusing the term "information" and they should really be referring to "energy" or "entropy" or something?

  • I agree with and think your last point is right on the nose. Outside of an observer, or the potentiality of being observable, I do not believe anything can qualify as information per se. Beyond our understanding of it? Absolutely, but this is no longer information to my mind. To qualify as information, as you have said, there has to be a potential for observation, or else it is just unintelligible disorder. Entropy is my word of choice. – INV3NT3D Jan 17 '17 at 18:00
  • 1
    My sense is that the meaning assigned to 'information' in physics is different than the meaning it is commonly used for. – virmaior Jan 17 '17 at 23:35
  • This depends on your definition of existence. Which definition are you using? – Carl Masens Jan 19 '17 at 0:21
  • I'm late to the party on this one but since it popped up due to recent activity: virmaior is absolutely correct, the issue is that Susskind, Preskill, Aaronson, and everyone else involved with what's going on in quantum information and its applicability to high energy physics use the term information in a specific way, different than how it is normally used. It has everything to do with Wheeler's it from bit argument, really the sole idea is that physics can be broken down to the point that information theoretic concepts become bedrock. – Not_Here Oct 14 '17 at 9:25
  • Susskind says as much during some of his lectures, I don’t remember exactly when but at some point he said something to the extent of “you can rephrase any question, no matter how complicated, as as series of ‘yes' or ‘no' questions if you’re willing to ask enough of them” and that’s the idea behind it from bit, any physical system can be described in terms of information theoretic bits (qubits actually). And one of the main ideas behind all of what is going on in this field is that the Bekenstein bound is the upper limit on the amount of information that can exist at one point in spacetime. – Not_Here Oct 14 '17 at 9:25

An objective, "mind-independent", view is a staple of physics, but so is the possibility of observation. The information lost in burning is not entirely unrecoverable, it is recoverable in theory, and some futuristic nanotech may be able to recover it, with a lot of effort, even in practice. If we do not go to extremes then it is well-known that information that was not scrambled radically enough can be recovered by crafty experts even today. Breaking the Humpty-Dumpty increases entropy, and hence "loses" some information, but Humpty-Dumpty can be put together again. This is not much different from acknowledging reality of the past, etc., we may never know what Aristotle did on his 20th birthday, but realists postulate that there is a fact of the matter as to what that was. As Dupre puts it somewhat sarcastically in Metaphysical Disorder and Scientific Disunity:

"...some set of facts that could be known that would permit the inference of the macroscopic from a sufficient knowledge of the microscopic. Perhaps we could not, even in principle, know these facts. But God, I suppose, would need merely to exist in order to know them."

But sarcasm aside, taking the "God's eye" view is typical among mainstream realists when interpreting claims that would not be sensible otherwise. E.g. in Relativity of Simultaneity and Eternalism Peterson and Silberstein invoke "Newton's god" (NG) to distinguish eternalism and presentism about time:

"There may be some who believe that NG is not a suitable tool for dealing with the presentist/eternalist distinction; in particular, one might findnd our NG question-begging since a god's eye point of view might somehow allegedly violate basic tenets of SR, however, one must note that by hypothesis NG is removed from the 4D-manifold (space-time) that she observes.... It would be absurd to argue, therefore, that two perspectives as different as these are, are in fact, metaphysically and empirically equivalent in principle; such a claim could only be sensible if one assumes a spatiotemporal-anthropocentric verificationism and there is no non-question begging reason to do so."

The second law of thermodynamics is an emergent statistical law that applies to macroscopic systems with large number of microscopic objects. It can be violated, and was observed to be so in some experiments, but the probability of that decreases with the number of objects involved. It does not apply to systems (taken separately) with influx of energy that prevents them from reaching equilibrium, e.g. to the gravitational accretion of matter that formed the planets of the Solar system. In short, there is plenty of room to validate the second law for all practical purposes, while maintaining whatever one pleases at the fundamental level. There is also a difference between physicist's and colloquial meaning of "information", a string of symbols gibberish to a human knower, is full of "information" in the sense of being correlated with the part of the environment that "created" it, but I do not think that this difference is relevant here.

Determinism is a bigger problem. It is a mathematical consequence of quantum mechanical formalism that unitary evolution by itself conserves information (more precisely, the von Neumann entropy). It is the non-unitary "collapse of the wave function" that creates problems, which is perhaps why most quantum information theorists, like Deutsch, subscribe to the Everett interpretations of quantum mechanics. There there is no collapse, and decohered states, which create an impression of it, can in principle recohere, unlikely though that is. Everett's is a strange "determinism" though, his branches realize every possible outcome rather than predetermine one, hence each branch "creates" itself through its past "choices". Only the global state is predetermined, but it is everything whatever anyway. Essentially, the unobservable branches are postulated so that there is somewhere for the "lost" information to go (or come from).

In statistical interpretations "collapse" amounts to observer specific reconditioning of probabilities, and so any information "loss" it entails will also be observer specific, it does not happen "objectively". Under objective collapse, and indeterministic interpretations of classical physics, one would indeed have to reject the conservation of information. If a ball rolls to the top of the Norton dome and then stops there, there is no recovering afterwards from whence it came and how long it stayed on top. This means of course that quantum mechanical formalism per se is neutral on the issue.

P.S. Time reversibility by itself does not entail determinism, and the Norton dome is a counterexample. Time reversed solutions to the ones mentioned above, when a ball sits on top of the dome for arbitrary time, and then rolls off in arbitrary direction, are also solutions (the shape of the dome makes equations of motion non-Lipschitz, and uniqueness of solutions is lost). What it does entail is that if information can be destroyed it can also be created "out of nothing", like the ball's elapsed time and direction of movement. The peculiarity of the dome is that self-initiation of motion requires no energy input here, not even an infinitesimal push. This is close to what free will libertarians want, free will acts as causa sui, and hence injects new information into causal chains. I should mention that many physicists consider the Nortom dome "unphysical", and "free will creates information" is likely to be as controversial as "consciousness causes collapse".

  • "which is probably why most quantum information theorists, like Deutsch, subscribe to the Everett interpretations of quantum mechanics." My impression was that Copenhagen was the dominant strain - even among QC theorists. – Alexander S King Jan 18 '17 at 17:26
  • @AlexanderSKing That's what I thought too, but turns out not exactly. After decoherence it is also unclear what Copenhagen is today beyond "shut up and calculate", and how it differs from statistical interpretations. Most prominent decoherence theorists like Zeh are also Everettians. – Conifold Jan 18 '17 at 20:35
  • I was going to mention the big bang, and less grandiose, spontaneous particle creation, as examples of information generation and therefore contradicting conservation of info - but my physics is too rusty to delve into that argument. Now that you mention it however - what about those two cases? – Alexander S King Jan 19 '17 at 1:28
  • @AlexanderSKing Carrol discusses interesting statistics on Copenhagen. The Big Bang is billed as "initial low entropy state" but there are "no-boundary" proposals where it would not be a temporal "creation". Creation/annihilation is just QFT analog of QM evolution, if it is unitary or not depends on your view of collapse physics.stackexchange.com/questions/193918/… – Conifold Jan 19 '17 at 1:39

The conservation of information means something like "the entropy of the universe remains constant over time". this is entirely compatible with the entropy of all of the individual systems increasing over time because the joint system of two or more entangled systems can have a state of lower entropy than its subsystems. For example, the joint system of two qubits in the state |0>|0>+|1>|1> has zero von Neumann entropy, but the reduced state of each of the subsystems has maximal entropy. You can look up how to calculate the entropy and the reduced state in "Quantum Information and Quantum communication" by Nielsen and Chuang.

Each of the subsystems in an entangled system has locally inaccessible information. The physical descriptors of each system depend on what you do to them, but their expectation value does not. For example, if you rotate a qubit its Heisenberg picture observables depend on the rotation, but their expectation value doesn't depend on the rotation. The only way to get this locally inaccessible information is to bring together all of the systems with which the qubit is entangled and do a suitable operation on them, see:

https://arxiv.org/abs/quant-ph/9906007.

There is no particular problem with such dependence existing even if nobody knows about it. This is an implication of testable laws of physics even though that particular fact is not testable.

You write:

From an empiricist or positivist point of view, this no longer qualifies as information, by definition.

Since empiricism and positivism are both false I don't see the problem here. Empriricism and positivism are both unavoidably ambiguous about what exists in reality because they do not and cannot have any clear explanation of what counts as an observation. For example, nobody can currently observe the core of the sun and maybe nobody ever will. So does the core of the sun exist? Also, nobody has ever seen a dinosaur, only fossils, so do dinosaurs exist? Empiricism and positivism both treat observation as a primitive: they are raw material to invent and prove ideas, but this makes no sense. Real observations have to be conducted by setting up a suitable physical system using an explanation of how the system works and it is adjusted until it is in a working state using that same explanation. Explanation is a result of a complicated chain of reasoning and can be used to test ideas but it can't provide you with ideas or prove them. The important issue is that the existence of such information is a part of the only existing solutions to problems. See "Objective Knowledge" by Popper chapter 1 and "The Fabric of Reality" by David Deutsch, chapters 3 and 7 and "The Beginning of Infinity" by Deutsch, Chapters 1 and 2.

There is another point of confusion in your post. The wikipedia definition of knowledge, which I think is the same as the standard philosophical view, is wrong. The idea is that information is distinguished from knowledge by conscious observation. This view totally neglects what is actually remarkable about human knowledge: it is highly adapted to solve problems. For example, the computer on which I am typing this is set up in such a way that if you make a change to it without understanding it you will break it. For example, if you change the design of its CPU or hard drive or any other piece of equipment without detailed knowledge of how it works it will turn into a useless piece of junk. But lots of non-human systems that existed long before people have the same property, e.g. - the chemical machinery in cells, the way eyes are put together and so on. So there has to be an explanation of this adaptation that applies not just to human knowledge but to the complexity in non-conscious systems like amoebae. The explanation is that both human knowledge and biological complexity arise by processes that involve variation and selection. It's appropriate to think of knowledge as being the property of being well adapted to solve problems, rather than thinking of it s well adapted information that happens to be in a human brain. For this reason and others Popper holds a theory of objective knowledge, much of which is not known by anybody. See "Objective Knowledge" by Popper and "The Beginning of Infinity" by Deutsch, Chapter 4.

Philosophers like to say subjective theories of knowledge are common sense, but they're wrong. By the subjective knowledge theory, any book or computer program or design of a complicated piece of equipment doesn't count as knowledge unless somebody happens to have memorised all the details.

  • 1
    I get the gist of your last section on adaptability and Popper's theory of objective knowledge. I don't entirely understand your first part: How does it resolve the issue of where the information contained in a book goes after the book is burned? – Alexander S King Jan 18 '17 at 17:41
  • The information in the book still exists even if it can't be measured: this is an implication of the laws of physics. Positivists and empiricists might say the information doesn't exist, but their positions are false and ambiguous. Measurements are interpreted in the light of explanations and getting them right requires effort and criticism, so measurements aren't primitives. Interpreting any measurement requires explanations, so anywhere you draw the line will exclude some current explanations. Nobody has a thermometer in the sun's core but we still know the temperature there. – alanf Jan 18 '17 at 20:28
  • So drawing the line between exists and what doesn't exist is ambiguous if it is done according to any criterion other than whether the supposedly existing thing features in an unrefuted explanation. See "The Fabric of Reality" by David Deutsch chapter 4 for more on this topic. – alanf Jan 18 '17 at 20:30
  • (I haven't played with quantum computing in ten years, thanks for dragging me back into it :-) ) -- Now that I think of it, the entangled state you used as an example is a little misleading because entangled states are when Von Neuman entorpy behaves differently from "normal" Gibbs entropy - Von Neuman entropy doesn't change in unitary evolution, whether for the whole universe or one qubit - so how is it related to the second law? – Alexander S King Jan 19 '17 at 0:17
  • The von Neumann entropy of the whole universe doesn't change. The von Neumman entropy of subsystems increases over time. The vN entropy of a single qubit does increase if it starts in a sharp state and becomes entangled under unitary evolution with another system. The entropy of the whole universe isn't relevant to the explanation of the possibility or lack thereof of decreasing entropy in any real experiment or interaction AFAIK. So I don't think there is a problem here. – alanf Jan 19 '17 at 11:22

Information exists independently of observation, but observation determines what information is known to us. If no living observer ever existed, the information present in the universe would still exist, just un-observed.

You destroy a note and that form the information took is gone, but the processes and events that created that information still exist to create the information again, perhaps in another form. If the information had the potential to exist once, then it has the same potential, within the dynamics of the universe, to exist again. This means that information is a bottom-up construction of fundamental information which we know as physics.

You can also reverse the process that destroyed the note and re-create the information. When you burn a note, every atom that comprised that note will react and behave in a determined manner, which is information itself. So if you take that information and follow it back to reconstruct the note, you have your original information. It will seem impossible to do, to follow and track every atom back to its position on the note, but the information to do so is there.

Hopefully, this will clarify what I am saying; "Hidden Information" is just information that cannot be observed directly, but can be constructed from "Available Information" which can be observed directly. These two forms of information compose the entirety of what information is to us as observers.

A systemic perspective (the right one, I believe), original ideas, no references.

  • Imagine a system that is formed of 26 subsystems, each one being able to prepend one letter to an stimulus after 1 second, all elements related (meaning that from A you can jump to any other system, A included).
  • We stimulate the system with an A entering through X and then wait. 13 seconds later, we get the pattern QKJHGBUYFVBEXA. This chain is able to express the sequence of RULES that have happened inside the system. Remember this word: RULES.
  • Inside this system, as I've formulated it, there are no deletions. Therefore, information is conserved, there is no possibility of loss.
  • You know ALL the RULES on this system, and you have a all "available information" (the characters chain). Then, you know this system, its history. There is no "hidden information", all is "available information". If you would receive a dot for any missing piece of information, and you get QKJ..B.Y...EXA, now there is "hidden information". That doesn't mean that the history of the system has changed. It only means that you cannot know part of its history.
  • Anybody, objectively, knowing the RULES, could deduce the history of the characters chain. even if you make a lot of stimulus at the same time, and you have many chains of characters, you can know the history of this universe.
  • Now, scientists sustain that nature is similar: there are no deletions in nature. Information is always conserved.
  • What are the RULES? We all know the set of RULES: they are called "causality". Causality tells you that if you stand in front of a famelic lion (action/stimulus from you to the lion: your presence), he will eat you (reaction/answer from the lion: you become a snack). Therefore, if you are there, somewhere, asking this, I can assume you didn't stood up in front of a famelic lion. Even if I die, that information is there, available for anybody that sees you. Just ask, of course, focusing on information, not on word plays or jokes.
  • Our knowledge of causality and chains is always limited. We don't know the chains, we don't know the rules. That is "hidden information". But using the small pieces of "available information" we were able to deduce how dinosaurs behaved. Perhaps one day we will be able to rebuild the information just deleted from a hard disk only by scanning the heat around the device.
  • Another way to see it is looking at any object. It is the consequence of a huge chain of actions and reactions. All information is there, regardless of we being able to understand it. There is some available info, and some hidden information. Any object is like a recorded sound tape of its history. But we don't have the mean to listen to most of it.
  • About entropy: entropy is a value that we can define about the state of any system. The state itself has no relation with entropy. Imagine, on our example system, that you define AAAAAAAAAAAAAA as a low entropy chain, but you've got QKJHGBUYFVBEXA, and this will be defined as a high entropy chain. On both cases, regardless of the entropy, the information is there at 100%: information is not related to entropy. But if you have obtained just QKJ..B.Y...EXA, ah, you've lost information, but entropy keeps exactly the same as on the case you have the information available.
  • About subjectivity and information: wikipedia states information ..."is a complete description of the thing, but in a sense that is divorced from any particular language"... I find it appropriate. Having said that, physical information gets a broader meaning that subjective information, the language you are able to understand. Subjective information can be "knowledge", but information is the chain of events of a system. QKJHGBUYFVBEXA is a representation of a system in our brain, but the sequence of events of that system is just a sequence of causes and consequences, independent of its representation. Do you see a cloud out there? That is subjective knowledge. But a cloud would be invisible, a cloud is just a regional light effect, diverting photons in different directions. All the sky has the same particles, but only some regions are able to divert light. What happens with light is what information is. What we see is just subjective data, an interpretation of it.
  • Black holes would be systems where information can be effectively lost inside (deletion, on our example). Odd to understand physically, with math is easier (I've heard :).

" nobody has ever seen a dinosaur, only fossils, so do dinosaurs exist? " The wrong is that we are supposing that is "existence" the parameter to express the rationality about what to say. Have really meaning talk about the information's existence? I think the concept of existence is ambiguous and that we don't know his deep and complete meaning. And i think that talk about the "existence of information" include the meaning of information ( as the definitions of both existence and information are information) so the concept is circular. I think the right question is "have meaning the communication of something without someone to communicate?" I think not. I think the right question is " nobody has ever seen a dinosaur, only fossils, so we are sure about dinosaurs information? "

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.