You say "probability is seen as a mathematical theory that we use to describe the physical world". I would say rather that probability and statistics are tools we use to help us test scientific hypotheses. To understand this let's go back a few steps and ask why we use probabilities.
Philosophers commonly distinguish between physical (or metaphysical) possibilities and epistemic possibilities. The first are concerned with what events happen or might happen and these are directly related to properties of the world. The latter are concerned with what we might reasonably believe to be true given our current knowledge. The dual of physical possibility is determinism; the dual of epistemic possibility is certainty. Probability can be thought of as a quantitative measure on possibility - a degree of possibility. Physical probability might then be thought of as the amount of possibility of some event happening, while epistemic probability represents the degree of credibility of some proposition given our available information.
To see that these are quite different, recall that probability theory was invented in the 17th century at a time when deterministic newtonian mechanics was believed to hold true of the whole universe. The fact that there are no physical possibilities in newtonian mechanics does not preclude us describing our state of information probabilistically. People playing a game of 'chance' can still talk of probabilities. We can even coherently bet on events in the past: for example, if there was a football match last week and neither you nor I know the result, we could have a bet on the outcome, even though this outcome is already physically determined. The odds that we agree upon reflect our assessment of the epistemic probability of the outcome given our current information.
If there are genuinely indeterministic 'stochastic' processes in the physical universe then there are physical possibilities as well as epistemic ones. Quantum mechanics is often understood to be indeterministic, because of the way the Born rule interprets the wave function probabilistically, though it is worth pointing out that some physicists do not accept the Born rule and consider QM to be a deterministic description of a multiverse. But whether or not there are physical possibilities, there are always epistemic possibilities: there are always propositions of which we are nearly certain, others less so, and others hardly at all, depending on what information is available to us.
In ordinary everyday usage we use epistemic probabilities all the time, though usually without much rigour. We may judge that it is highly probable that the defendent is guilty given that his fingerprints were on the murder weapon, the victim's blood is on his shirt, a security camera shows him fleeing the scene, and witnesses heard him threatening to kill the victim. We may judge it highly improbable that Johnny really did his homework considering that he has used the "a dog ate it" excuse many times, his clothes show evidence of him playing football, and there are no dogs in the neighbourhood.
The harder question to answer is, in the context of scientific method, is it helpful to use the concept of epistemic probability to describe how credible some scientific statements are versus others? Here we get disagreement. At one extreme we have Popper who in "Logic of Scientific Discovery" maintains that we cannot speak of the probability of a theory or hypothesis being correct - that a theory would always have probability zero, though in his later writings he allows that we can reason probabilistically about hypotheses, but such reasoning is still deductive and not inductive. At the other end we have Edwin Jaynes who in his book "Probability Theory: the Logic of Science" argues that we can understand scientific inference as an application of Bayesian probability theory. I suspect most scientific practice lies somewhere in between. We use probabilitistic and statistical methods to help test hypotheses, but we do not usually speak of the probability of a theory being true.
Your question, "is probability a product of the scientific method, or is it a part of it?" suggests a false dichotomy. Probability is a tool that has evolved to help us address problems in scientific inference. As such it is both the product of scientific progress and a part of it.