Why does evidence increase the probability of a particular hypothesis being 'correct'?

Question

As we accumulate more evidence to support a given hypothesis we have increasing confidence that the hypothesis is 'correct'.

How do we justify this?

Answer 1

It's sort of circular, as you would define "evidence that supports a hypothesis" as "things which would increase our confidence in the hypothesis."

To elaborate a bit:

• You could see for example the Sunrise problem. Laplace's Succession rule shows that the more times you see the sun rise in the past, the greater confidence you have that it will rise in the future.
• More generally, you can apply Bayes' Theorem to update any hypothesis you have. If you know the probability of evidence occurring given your hypothesis, then your posterior belief in the hypothesis' correctness is directly computable from this and your priors
• VC theory proves that "simpler" hypotheses are more likely to be correct. Postulating a world in which evidence coheres with "universal laws" is (almost?) always simpler.
• The Law of Large Numbers indicates that repeated experiments will tend towards truth. This is discussed somewhat more here.

Answer 2

We justify it by calling it the "scientific method" which as been around since the western age of modernity and is the basis of all modern science. We propose a hypothesis based upon a hunch, a reason, or observation. We then look for additional 'evidence' or facts to justify our hypothesis. If we can predict a future chain of events based on our accumulated evidence we call it a theory. Given enough evidence we may call it a law. Law is a mental shorthand to explain a series of phenomena; but law as an entity does not exist. We use the word to express the regular succession of certain occurrences in the phenomenal world - for example, the Law of Gravitation.