How do we introduce the use of undefined terms and why are we allowed to do so? What is the difference between setting the rules of usage/manipulation and actually giving a definition?
"How do we introduce the use of undefined terms?"
In a formal context (such as logical foundations of mathematics) this can be done with implicit definitions. The locution "implicit definition" refers to the idea that a term is not defined by an equivalent relation, but used in axioms, and that the axioms somehow "define" the meaning of the term. So for example, "bachelors are unmarried men" would be an explicit definition because it's an equivalence relation, whereas "number" in Peano's axiomatisation of arithmetics is never explicitly defined, but used in the axioms, and a number is taken to be something that satisfies the axioms of arithmetics.
The idea can be extended from formal languages to the terms of scientific theories. Notably, force and mass in Newtonian physics can only be defined on the basis of one another, so that some authors view the axioms of the theory as implicit definition of its terms.
It can also be extended to natural languages, and some, such as Wittgenstein, went as far as claiming (roughly) that meaning is use. But even without emphasis on usage, some people think of meaning in terms of direct reference to natural properties or kinds, and in this context, no explicit definition can be given a priori, because the term acquires meaning by direct ostentation (for example, a tiger is "this kind of animal" (while pointing at a tiger)).
"What is the difference between setting the rules of usage/manipulation and actually giving a definition?"
One formal difference is this: in the case of explicit definition, the term can be replaced by its definition in every occurrence without any change in meaning (think of bachelor and unmarried man), whereas this is not possible with implicit definitions.
In the context of philosophy of science, this plays a role for realists who want to argue that a term such as "mass" or "species" still refer to the same properties even when theories change (this is combined with the idea of meaning as direct reference mentioned above). One can argue that the aim of theorising is to find essential characteristics of real properties, which will take the form of revisable implicit definitions, and that a certain continuity in theories is good enough to claim that we're still "taking about the same properties". This would be more difficult to maintain if all terms were explicitly defined.
More generally, if there's no explicit definition, operationalist epistemologies (the idea that the content of scientific theories strictly reduce to operations and observations) are difficult to sustain.
So the consequences are quite important.
"why are we allowed to do so?"
In a formal context, the rationale for this is that explicit definition of a term must use more basic terms--but how shall we define the most basic terms? Implicit definitions make it possible. This is why we are "allowed to do so": because there's no other option! This is clear in the case of numbers. The same reasoning can probably be extended to scientific contexts, where we need "fundamental properties" to start with.
In natural languages, one rationale for this is that dictionary definitions are always a posteriori: they attempt to make explicit how words are used, but usage comes first. Quine observed this, and claimed that explicit definitions can only be given in very particular cases (when a new technical term is introduced explicitly), but that most dictionary definitions are more like attempts to capture usage.
Another possible rationale could refer to all arguments in favour of direct reference, such as the fact that all we think we know about a term (that gold is yellow...) is in principle revisable by experience. Those arguments were put forth by Kripke. Arguably, if they are valid, then no explicit definition can be given to a term, or at least, not a priori.