The main 'problem' with (non-transfinite) induction is that the proof itself that would cover all the numbers is not completed in a finite length unless one includes a rule that is implicitly infinitely long.
The proof via induction sets up a program that reduces each step to a previous one, which means that the actual proof for any given case n is roughly n times the length of the stated proof. The total proof, to cover all cases is then implicitly infinite in length. But for any case that comes to hand, we could do the scutwork and spit out a proof that is still finite. So does a collection of infinitely many finite proofs count as a proof? Well, that depends on why on earth you think there is an infinite set of proofs to begin with. Either you think there is a complete, infinite set of integers, or you don't. Either way, induction it true, but it is true for a different reason in each case.
In Classical mathematics, the question seems very stark: you either need the 'induction schema' which is either second-order, and thus potentially self-referential, or has infinitely many entries. But this is because classical mathematics is not 'constructive', it always models time by taking up space, by creating copies of objects, rather than by taking up time.
If you need something to be true of any integer, but your statement systematically differs from integer to integer, so that there is not a single closed form that captures it, then you have to make infinitely many copies of it, and modify each copy. This makes induction seem strange. But it all takes care of itself. If you think the set of numbers exist, there is no reason why your infinite set of axioms should not exist, one for each number.
From a constructivist point of view, mathematics should be represented not by true statements but by iterative constructions. A proof should be a program that runs on inputs and verifies that an assertion can be reduced to agreed principles. Then every proof shares that form with an induction. We very seldom ever write out a proof proceeding entirely from axioms one step at a time. Instead, we invoke principles and forms that indirectly indicate how the 'real underlying' proof would be assembled.
And selecting a number does not require embodying the set of all numbers so that you can then choose one. It just means what it says, if I counted up to someplace and just stopped anywhere, I would have chosen an arbitrary integer without vouching for the existence of a set of all integers. Our notions of infinite sets are just tricks of language, not depictions of infinite objects.
So issues of infinities do not have to enter into the consideration here. You only need to accept that iteration is a reasonable thing to do. You can do that within your definition of what a construction is, and therefore within the definition of what it means to prove something. In that case, you do not need an infinite axiom base to support proofs with loops.
For Kleene, a proof is a recursive function that computes a truth value, for Bishop it is an iterative process that validates an assertion. Both of these automatically include induction as a natural form of proof.
Beyond this, there are extreme forms of objection to repetition. The idea that there is a natural limit on the human ability to keep track of things and that we should not allow arbitrarily long processes into our plans. This reaction arises quite naturally in the proof of the 4-color theorem, where there are over a thousand separate cases, and each case is an algorithm that reduces a complex structure to one of the earlier cases. It is a proof that no human could ever possibly comprehend, and our faith in it relies upon our understanding of the engineering process that selected those cases and automatically wrote those algorithms through brute force enumeration. It makes one worry.
But in such a framework no general mathematical truth is ever really true. A notion like the idea of induction is impossible to state, and so is a simple theorem like the idea all maps can be painted in four colors. There are surely maps too complex for a human to paint them, so surely there is no way of knowing about that. To me, this is nonsense. We have invented computers, and they collectively manage exobytes of information. We have more than enough power to handle mathematical truths far beyond any individual's or group's ability to fully comprehend them, and yet we don't doubt that we can enforce the rules that handle that information. This philosophy has to just crumble under observed reality.
