Your question is discussed among mathematicians under the key words Platonism and constructivism. Platonism is the opinion that everything is already there (like Plato's ideas). Constructivism is self-explaining.
An implication of Platonism, i.e., that mathematics can be discovered, is transfinite set theory. There it is assumed that fro instance all natural numbers and all real numbers are existing. Since it is impossible to construct or identify or distinguish all uncountably many real numbers, this opinion would be meaningless in constructivism.
Cantor as the founder of transfinite set theory had a clear vision of the existence of all numbers. Of course, if existing, they cannot exist in some corner of the universe but only in the intellect of God. Same is valid for the hierarchy of infinities playing the dominant role in transfinite set theory.
"Every single finite cardinal number (1 or 2 or 3 etc.) is contained in the divine intellect (from St Augustin: De civitate Dei, lib. XII)." [G. Cantor, letter to I. Jeiler (20 May 1888, Whitsun)]
"Dominus regnabit in infinitum (aeternum) et ultra." {{The Lord rules in eternity and beyond, from Exodus 15,18.}} [G. Cantor, letter to R. Lipschitz (19 Nov 1883)]
"Compare the concurring perception of the whole sequence of numbers as an actually infinite quantum by St Augustin (De civitate Dei. lib. XII, chapt. 19) [...] While now St Augustin claims the total, intuitive perception of the set (n), 'quodam ineffabili modo', a parte Dei, he simultaneously acknowledges this set formally as an actual infinite entity, as a transfinitum, and we are forced to follow him in this matter." [G. Cantor, letter to A. Eulenburg (28 Feb 1886)]
What would be some implications of mathematics not existing and to be discovered? Transfinite set theory would be a meaningless game because nearly all of its objects would not exist and could not be defined. Of course set theorists will not agree to this argument, at least not in public.
To deal with this from a philosophical point of view, I can recommend Cantor's philosophical writings, which however are in German or French: G. Cantor: "Über die verschiedenen Standpunkte in bezug auf das aktuelle Unendliche" and "Mitteilungen zur Lehre vom Transfiniten", Ztschr. f. Philos. u. philos. Kritik vol. 91, p. 81 - 125 (1887); vol. 92, p. 240 - 265 (1888).
Some important paragraphs have been translated by Mückenheim into English: See Chapter IV of https://www.hs-augsburg.de/~mueckenh/Transfinity/Transfinity/pdf.
