Order/disorder and complexity

Question

What is the relation between order/disorder and complexity ? Sometimes I found the terms confusing and ambiguous.

And higher entropy implies low complexity, does not implies low entropy implies higher complexity.

Answer 1

What I think others miss here is that entropy has a precise, exact definition relative to models of physical phenomena. Indeed, the answer by CriglCragl seems flatly wrong on the basis of my statistical mechanics classes.

Entropy is defined in a thermodynamic contexts: ones where the interesting "motion" your physics studies is primarily not that of individual particles but of a very large number of particles as a whole, and the changes in aggregate quantities characterizing the collective behavior of the system (opposed to the variables characterizing the individual behavior of the particles). Of course, the collective is composed of parts, but there is information about the parts that is irrelevant to the behavior of the whole (particle 2 is at position P at time t), and information about the whole that is not informative or even well-defined for any one of its parts (the standard deviation of the velocities of particles in the system is 0.3 m/s).

This dichotomy is expressed in the notion of a microstate in contrast to a macrostate: the former is a particular configuration of a system, with omniscient knowledge of individual behavior of all particles it's comprised of, whereas the latter is a particular configuration of a system with respect only to its collective behavior. Exactly what a microstate is is a matter of the ambient physical theory: in classical mechanics, a microstate is a particular choice of every degree of freedom of every particle in the system; in quantum mechanics, it'd be the composite wavefunction of every system. A macrostate is simply all microstates that are "macroscopically the same," according to some particular macroscopically-measurable quantity (usually total internal energy).

The entropy of such a system, is a specific, well-defined, number that measures how many microstates could have caused a particular macrostate: in general, it is defined to be proportional to the sum of the product of the probability that a particular microstate is occupied with the logarithm of the same (really, Stack Exchange Philosophy, no MathJax? How is any good philosophy to be done without LaTeX?). In systems where all microstates are equally probable, and there are a finite number of total states, this can be simply expressed as being proportional to the logarithm of the number of microstates per macrostate.

How does this correspond to the colloquial notion (using Gibbs' neologism: "mixedupness")? Well, taking the multiplicity definition, system energies with fewer possible microstate configurations tend to have all particles congregating in a particular state (e.g. "all particles stopped," "all particles moving at 5 m/s equally-spaced along the circumference of a 1-meter-radius circle") that seems heuristically "more ordered." Precisely: high-entropy states yield less information about their microscopic configuration than low-energy ones (which leads to Shannon's definition in information theory). A high-school chemistry teacher, to shut up my foundational questions, once defined it as "local, spatial homogeneity;" this is not a definition of any sort, but clarifies how the colloquial notion tends to present itself in practice.

Many intuitions at this level tend to break down. I'm not sure of counterexample systems to the "disorder = high-entropy" notion, but an instructive one comes in the case of temperature: once one defines temperature in terms of entropy, there exist negative-temperature systems that are hotter than any known systems. Since the notion of "disorder" is indeed poorly-specified, I don't know how to search for such an example, though.

Answer 2

We consistently find that complex dynamics occur at points between a system being at the high entropy end of it's options, and the low end. Like when you mix cream into coffee, it's the intermediate state before full mixing has occurred that you get interesting swirls. Discussed here: How can nature without self-awareness and intelligence create living beings with self-awareness and intelligence?

Entropy is an interesting topic, not so much because it is difficult, but because it challenges our intuitions and requires scrupulous and clear thinking to interpret in practice. For instance, even many physicists think of entropy as a property a system has, but it's actually relative, a system only has an increase or decrease relative to itself at a different time. This is important because not all the dynamics of a system may be visible, for instance at a given energy level.

Order, disorder and complexity, are ordinary words with intuitive meanings in ordinary conversation. But in physics, they have proved key to thinking about thermodynamics, and the arrow of time, and complexity and chaos have special more precise definitions in their own theories, and have helped lead to the powerful idea of emergence, and of emergent dynamics.

See

How does entropy explain consciousness and the forward direction of time?

Have philosophers speculated on how chaotic forces meeting together can result in order?

Why is a measured true value “TRUE”? (about Shannon or signal/message entropy)

Answer 3

I'm no good at math, but I feel I can convey the meanings of the pairs order-disorder and simplicity-complexity via numbers.

1. Order: 1, 2, 3, 4, 5, ... There's some kinda rule that determines this sequence (+1 in this case). We're able to, if you notice, predict the next number.

2. Disorder: 3, 2.0073, -3089, ... ? Randomness essentially. We can't predict the next number in the sequence

3. Simplicity: The sequence in 1 (vide supra) is simple (just add 1 to the preceding number)

4. Complexity: What about a sequence such that the nth term, Tn = (n^2 + 3^(n + nth prime)/(sqrt(2.3078 + ((n + 2/n)th Fibonacci number)

Answer 4

And Higher entropy implies low complexity, does not implies low entropy implies higher complexity.

I think @MarcoOcram has provided a clear distinction between order/disorder andsimplicity/complexity pairs. However, the entropy refers to neither of them, but rather to certainty/uncertainty - where high/low entropy means respectively high/low level of uncertainty. Information results from the reduction of uncertainty - it is the negative entropy.

E.g., if I have tossed a coin, without showing you whether it landed with the head or the tails up, your uncertainty about the result is 1 bit. Once I you learn the result, the uncertainty vanishes, and we say that you gained 1 bit if information.

Answer 5

Classically, the world was thought to divide into a binary of orderly-and-simple (squares, circles, Greek architecture) and chaotic-and-complex (the ocean, the weather, war). But we now have a more nuanced understanding of the world.

A good way of understanding complexity is as a measure of meaningful information.

• Order: A circle is simple and orderly. We can describe it using very little information (a set of points in a plane, all at the same distance from the center point), but that information is highly significant.

• Randomness: Now consider a group of random dots. They take a lot of information to convey (you have to individually describe the position of each dot, since there's no pattern) but that information is not very significant. There's nothing that knowing the exact position of the dots gains you over knowing that they are randomly distributed.

• Complexity: Complexity is a large amount of significant information. A photograph is complex. You can't describe it easily, but the position of each dot makes a significant difference.

• Chaos: The new(ish) mathematical science of "chaos" describes a special kind of complexity that can often include elements of both randomness and order. A chaotic system is typically unpredictable at one level of scale, and highly predictable at another. The weather is a archetypal example of a chaotic system.

So what's the relationship with entropy? Entropy can be conceptualized as a loss of meaningful organization. When entropy is high, randomness is high, but complexity is low.

For some good beginning primers on the subject I recommend Chaos (Gleick) and Complexity (Waldrop).

Answer 6

Complexity refers to the degree to which a system or subject has many parts interacting in a variety of ways. Orderliness refers to the extent to which the parts of a system or subject or organised in some meaningful way. Simplicity and complexity are therefore not correlated in a fixed way with order and disorder.

Take a relatively simple system such as a set of Lego blocks. If you have children who scatter them on the floor, you will consider the blocks to represent a disordered system. If your children assemble the block to form a passable model of the Parthenon, you will consider the set to be in a more orderly form, although its complexity will not have changed.

Your observations about entropy are somewhat misguided. Entropy refers to the extent to which energy is dissipated through a system, rather than to states of order and disorder per se.