The best way to understand Popper is to read Popper. There are a few commentators who have correctly understood his ideas, but the vast bulk of commentary on Popper is not even able to state his ideas correctly. Lakatos, Feyerabend and Kuhn are especially bad and should be avoided.
To understand falsification properly, you need to understand Popper's theory of knowledge more broadly. Most philosophers of science who take science seriously and think it is good are inductivists: they believe in a process called induction. Induction supposedly involves (1) taking observations, (2) using them to make theories, and then (3) showing those theories are true or probably true by more observations. People have looked at many phenomena such as the night sky, biology, medicine and so on, without learning much for thousands of years. So just observing stuff doesn't do much good. If you don't know what to look for, just observing will not produce progress, so step (1) is impossible. In addition, explanations don't follow from observations. The theory of stars has implications for many events we will never observe, e.g. - supernovae that took place before there were human observers, and those events don't follow from observations without a theory of how stars change. So steps (2) and (3) are also impossible.
So if we don't get theories from observation how do we get them? We guess. You look for a problem: some issue that is not explained by current ideas. You guess solutions to that problem. You then criticise the proposed solutions. This criticism may involve experiments, but many theories can be eliminated without doing experiments, e.g. - inconsistent theories.
An experiment involves looking for a situation in which two or more different ideas about how the world works make different predictions. You then either set up that situation or look for an existing system that realises that situation. Newton's theory of gravity and Einstein's general theory of relativity made different predictions about Mercury, and Newton's theory was refuted.
Some philosophers make a lot of fuss about the possibility that you might do an experiment wrong or misinterpret the results. But as Popper pointed out in Logic of Scientific Discovery, Chapter V (especially Section 29), this problem is solved by his epistemology. If an experiment contradicts an existing theory, that's a problem. This problem can be solved by any guess that explains the difference and is not eliminated by some criticism. The discovery of Neptune was taken as an example above, so let's look at it. An unsolved problem was found in explaining the orbits of some planets. Urbain Le Verrier guessed that there might be another planet. He worked out some constraints on where the planet could be to produce such effects, Johann Gottfried Galle looked for it and found it. If Galle had not found the planet that problem would have remained unsolved. Perhaps some other explanation could have been found to reconcile Newtonian mechanics with observation, perhaps not. Popper recommended that a proposed solution to a scientific problem should be rejected if it was ad hoc: if it had no implications beyond the problem it was invented to solve.
I'm going to skip the heliocentric theory because it is fairly similar to Newtonian mechanics. If you want a long list of examples, see the introduction to "Realism and the Aim of Science" by Popper.
Mathematical theories are about abstractions. They can be critically discussed, but not experimentally tested. 1+1 = 2 even though it is possible to think of examples of putting two objects together and only getting one object as a result. If you move two piles of sand together, you may only get one pile. So you have to think carefully about what systems you take as models of mathematical operations such as addition. For a discussion see "Realism and the Aim of Science" by Popper Chapter III, Section 24.
As far as probability is concerned, the best existing explanations have been provided by David Deutsch, see
For explanations of Popper's positions, see "Objective Knowledge" by Popper, Chapter 1, "Realism and the Aim of Science" by Popper, "Logic of Scientific Discovery" by Popper, "The Fabric of Reality" by David Deutsch, Chapters 3 and 7, and "The Beginning of Infinity" by David Deutsch, Chapters 1,2,4 and 13.