This question seems to either be at the forefront or the background of countless philosophical enquiries. Much has been written on Wittgenstein's rule paradox (e.g. Kirke's Wittgenstein: On Rules and Private Language," as well as many others). Here is Wittgenstein's rule paradox:
§201. This was our paradox: no course of action could be deter- mined by a rule, because every course of action can be made out to accord with the rule. The answer was: if everything can be made out to accord with the rule, then it can also be made out to conflict with it. And so there would be neither accord nor conflict here.
It can be seen that there is a misunderstanding here from the mere fact that in the course of our argument we give one interpretation after another; as if each one contented us at least for a moment, until we thought of yet another standing behind it. What this shews is that there is a way of grasping a rule which is not an interpretation, but which is exhibited in what we call "obeying the rule" and "going against it" in actual cases.
Hence there is an inclination to say: every action according to the rule is an interpretation. But we ought to restrict the term "interpretation" to the substitution of one expression of the rule for another.
§202. And hence also 'obeying a rule' is a practice. And to think one is obeying a rule is not to obey a rule. Hence it is not possible to obey a rule 'privately': otherwise thinking one was obeying a rule would be the same thing as obeying it. (Wittgenstein, Philosophical Investigations)
I guess the question is: how do we know how to apply some rule we learned to something unknown. Why are we certain that 13 x 13 = 169 and not 196 (if we’ve never gone past 12 x 12 before)? --As in, how can past experience help us if what we are applying the rule to is something unexperienced? It could seem as though we need to follow a rule to follow a rule (an infinite regression). That is, there would be a certain rule for applying rules by some similarity of circumstances; of course, it gets somewhat tricky for something like mathematics--where we often change the elements of the system to build foundations (e.g. the change in the concept of number that Dedekind and Cantor brought about), and thus, are still expected to follow certain rules in new (but perhaps analogous) circumstances. (To identify situational analogs seems to be another sort of rule following.) The point is that it seems our minds "fill in the blanks," so to speak, of what is unknown by what we know. But it's also unclear why there is certainty when we apply a rule to new sets of problems.
The question is not only "how do we know how to follow a rule," but also how do we know that when we follow a rule we are using it correctly--or even if we are using it correctly, in the same sense that we learned the rule, how do we know that what we are calculating behaves (consistently) according to the rule? There are some analogous situations outside of mathematics in which we seem to follow either a learned, experienced, or innate, rule of perception: some colors that the human eyes sees are not spectral colors (e.g. pink, tan, greys etc.), and thus, don't correspond to a single wavelength on the color spectrum.