This really belongs in the stats stack exchange (a.k.a. cross validated). But I'll give you an answer here.
Before getting into any math, I'll just say that there's a problem of epistemology here. You can't ever know anything about the real world with perfect confidence. The only thing you can ever know with perfect confidence is whether the axioms imply the conclusion. This is what a "proof" always consists of. That is, you can prove that "if A then B", where A is the set of axioms and B is the conclusion. You are not 100% confident in A, but you are 100% confident that A -> B. Premise A, itself, may have been the result of another proof, which means that you are 100% confident that A is true if the premises that led to A are true. If you follow the chain backwards, you eventually get to the original axioms, which were either a claim about the real world (which you cannot be 100% confident in) or a definition (usually in math). So any conclusion you make from a chain of proofs is either one in which you're only as confident in as the original set of axioms, or it's tautologically true. For example, if I posit C as an axiom of which I'm 70% true, and I prove that C -> D then, assuming this is a valid proof, we are 100% confident that if C then D. However, because we are only 70% confident in C, we are consequently 70% confident in D. However, instead of a true proof, I might have made a probabilistic claim. That is, C -> D with 90% confidence. In this case we are 63% confident in D, because 90% of 70% is 63%.
We can always form hypotheses and conduct experiments that are true tests of the hypothesis. This procedure can be used to increase our confidence in a claim, even push it to near-certainty. But it will never be certain.
The consequence of this fact is that we are not certain about anything, even our best scientific theories. We merely get to a point that a certain claim is so negligibly unlikely to be wrong given the set of data we currently have testing the claim, that our best bet is to accept it. If you google "most accurate theory", you will find resounding results that it is QED (Quantum Electrodynamics). But how can there be a "most accurate theory" if we can know that certain theories are true? Well, we cannot. However, QED makes certain statistical claims on what you would observe in a particle collider, were it true. From there, you can calculate the probability of observing the things it claim, were it not true (by random chance.) In QED's case, the probabilities of its claims across the board were generated by chance, by non-QED hypotheses, are so dramatically low that not believing in QED is as bad an epistemological bet as you can make. But it's not guaranteed to be wrong with 100% certainty.
You can apply this to anything involving probability. So, for example, we might come up with a probability that the sun will rise tomorrow, given the set of historical observations. A frequentist would predict 100%, and a Bayesian would predict something so close to 100% that the difference is negligible. But we cannot know for sure. Assuming we have not yet discovered celestial mechanics, we can make inferences on the probability of the sun rising given previous events.
Now for a more grounded example, to help cement this concept. Suppose we take a coin. We can assume it's a "fair coin" (i.e. 1/2 chance of landing on each side) but we don't know that for sure. (In fact, it's probably not exactly true for any coin because there are subtle imperfections that make it not a perfect thin cylinder.) However, its probability of landing on heads is a purely empirical claim. We can do something like a hypothesis test, which is similar to what I described in QED, but much simpler.
To give you an idea of how a hypothesis test works, consider this. Suppose we start with a coin, and ask "is this a fair coin?" (i.e. 50% chance to land on either side.) One thing we can do is flip it n times and ask "what is the probability that a fair coin would have been at least this deviant from the expected result?" The expected result of n flips of a fair coin is n/2 heads, but we know that it can deviate from that. There's nothing abnormal about that. When I say "at least this deviant", what I mean is that the number of heads is at least as far from the expected result as we have observed. So suppose, for example, that we flip it 100 times, and we get 55 heads. To be "at least this deviant" simply means outside of the range of 4 heads away from the expected result of 50. That is, below 46 or above 54.
Suppose we flipped it four times, and it landed on heads only once. The probability of being "at least this deviant" is the sum of the probability of landing 0 heads, 1 heads, 3 heads or 4 heads. It turns out that the probability is 62.5%. That is, a fair coin would have a 62.5% chance of being at least one off the expected result of two heads. That's pretty high. So is this grounds to reject the "fair coin" hypothesis? No. But that doesn't mean we should blindly accept it. We have to keep going.
So now we flip it 10 times, and we get 2 heads. This deviates from the expected result by three. What is the probability of deviating from the expected result of 10 flips by at least 3? The answer is ~10.9%. That's not very low; things of that probability happen all the time in our daily lives. Should we reject the fair coin hypothesis on this basis? It would be a weird standard if we did.
Now let's say we flip it 100 times and get 30 heads. We have deviated 20 from the expected value. The chances of deviating at least that much is ~0.003%. That is, a fair coin has roughly a 3 in ten thousand chance of it's "head-count" in 100 rolls to deviate by at least 20 from the "fair coin" expected value of 50. But this coin did. This should constitute "strong evidence" that this is not a fair coin. Now, does that mean it definitely isn't? Well, I mean, the chance of a fair coin landing like that is not 0. So it's still possible that it's a fair coin. But given the result we just observed, it would be a fairly bad epistemological bet to conclude that it's fair. Some people say "we should reject the hypothesis" because the result was so low. I, personally, don't take such a binary view, but I would say that I'm so confident that it's not a fair coin that I will act as though I'm certain (even though I'm not). After all, that's how we treat epistemology in many aspects of every day life. Just keep in mind that, with more rolls, we can become even more confident in our claims. But never 100% confident.
Then the question is, what do we conclude about the coin's probability of landing on heads? Well here's the tricky thing about adjusting our views based on evidence. It's far easier to make a negative claim than a positive claim. That is, if we take a theory (e.g. "this coin is a fair coin with a 50% probability of landing on heads"), the best we can do with that is formulate an experiment to see if the claim of this theory holds up. If it does, we haven't exactly confirmed the theory (other theories could make the same claim) but we failed to reject it. However, if the claim appears to be unequivocally false, we can easily reject the theory. So the question of "after the coin experiment, what do we choose to believe is the true probability of landing on heads?" is far harder to answer than "is the probability of landing on heads an a priori given value?"
There are several ways of addressing this problem. Bayesian statistics is one such novel way. Instead of claiming "the probability of landing on heads is x", we instead have a probability distribution for the value in question. Then, as we obtain evidence from new experiments, we adjust that distribution. In the coin example, we might say "given no knowledge, we'll assume that the probability of landing on heads is equally likely to be anywhere between 0 and 1". That's our probability distribution. Then we conduct an experiment by flipping the coin. Every time the coin is flipped, our distribution shifts slightly. After, say, 100 flips, if we have 30 heads, our distribution would be somewhere centered around 0.3, but it's not fixed at that value. The more times we conduct this "experiment" the more confident we become in our assessment, and the narrower this distribution becomes. (Recall that flipping 2 out of 10 heads wasn't enough for us to reject the fair coin hypothesis, but flipping 30 heads out of 100 was more than enough.)
This goes to something called the "Law of Large Numbers", on which we're relying to pan out to the truest answer in the limit of many experiments. You can look it up, but loosely speaking, it claims that if an event is drawn from a probability distribution, as the number of times we generate this event approaches infinity, the distribution of the outcome of this event approaches the true probability distribution. So, if we perform the experiment a few times, there's no guarantee that its distribution will look anything like the true distribution. But we expect it to look more and more like that distribution as the number of experiments grow. In the coin example, we would often expect a fair coin to give something like 2 out of 10 heads, but we would almost never expect it to give 30 out of 100.
Going back to the casino discussion. If we take a casino game of pure chance, involving dice or coins, once we learn the parameters of that game we can easily prove that the game favors the house using simple math. However, as I mentioned in my first paragraph, that proof has axioms. So we're only as confident in the conclusion of that proof as we are in the axioms. Most of the time, those axioms are that the tools of the game are fair tools. (Fair dice, fair coins, etc.)
