When reading Kant's CPR It is always important to remember that he is responding to Humean skepticism. He accepts Hume's argument against absolute knowledge. He is attempting to formulate a ground for any kind of objective knowledge whatsoever.
He assumes that mathematics can be the source of such a ground. Hence, he begins with expositions for time and space. Moreover, given that skepticism essentially attacks the premises of logical arguments, he attempts divorce mathematics from logic. But, he introduces the symbiotic condition that what is sensible must be made intelligible what is intelligible must be made sensible.
His attempt, however, is well-grounded. Kant criticizes the principle of the identity of indiscernibles long before modern mathematical logicians had rejected it when formulating first-order logic. Kant relegates the principle to logic while attributing numerical difference to spatial intuition. If you look up numerical identity in Strawson's book "Individuals," you can get a sense of how skepticism is the underlying factor driving all of this. Philosophers use the term "reidentification" to speak of how we accept a perception of an object moving in and out of our field of vision as being singular.
When considering Kant's references to space and time, it is useful to keep in mind the trichotomy consisting of syntax, semantics, and pragmatics. Syntax studies the signs of a language, semantics studies the possible meanings those signs may have, and pragmatics studies the relationship of a language to language users. The sad state of affairs that exists in the foundations of mathematics stems from the intractable positions of analytical philosophers and logicians against properly addressing pragmatics. Kant's assumption that mathematics can ground objective knowledge is a matter of pragmatics rather than the metaphysics at the heart of objections. And this is what must be understood when using Kant's explanation of time and sense as conditions of sensibility.
If Kant had not introduced a distinction between phenomena and noumena, there would be no object corresponding with the phenomena of sensible intuition. Then, critical philosophy would degenerate into solipsism. He is not denying an external reality. He is simply conforming with the constraint skepticism places on our ability to assert knowledge of that reality.
One problem with your question is that you are overlooking Kant's statement that matter is "that in appearance which corresponds to sensation." So, whatever is to be called "material" is to be correlated with appearances through sensation. And a page or so after your quoted passage he reiterates that "matter" only refers to the sensations attached to outer substances and not those substances themselves.
When I think about Kant, I find it useful to consider "indispensibility arguments" for the existence of abstract objects. My original scientific interest had been biology. If the theory of evolution is essentially correct, how does mankind have any facility through which to know the material truth of reality? I certainly have no evidence that mathematics is anything but a product of human experience. And, formalism in the strict sense of analytical philosophy maintains that necessity, even that of mathematics, follows from stipulated rules. Consequently, there is the joke about mathematicians be platonists except on Sunday.
Were to give credence to assertions commonly made in scientific publications, I would seem to live in a world where mathematics proves the existence of dimensions I cannot witness in any way. Indispensibility arguments ask people to believe in things they cannot witness for the sake of believing in a reductionist philosophy. The next time you hear that human thought can be reduced to Turing machine intelligence, insist upon the explicit details. It is a belief about science without merit.
And, laughably, the academic community who rejected Kant used arguments denying the existence of mathematical objects. Now, for the sake of reductionism (and atheism in many cases), their inheritors use arguments asking people to believe in such.
Before his logicism, Bertrand Russell proposed that projective geometry might be a candidate for Kant's "pure geometry." Projective geometry is ubiquitous in the mathematics of physics. The truth tables of classical propositional logic satisfy the axioms of a finite affine plane with an associated with a projective plane proven unique up to isomorphism. Modern notions of foundations would reject this basis because of circularities.
But, if knowledge of external objects is restricted by skepticism to the relational form of sensory perceptions, it would seem that making sense of a circular ground is required. Leibniz happened to believe this to be possible. Hilbert's popular view of mathematical existence is incompatible with this.
