Regarding your first question : "Can numbers start from something trivial such as 0, 1 and then reach infinity?"
This is a mathematical question and the mathematical answer is no. To understand why the answer is no requires some philosophical considerations.
To best way to understand why is to consider Cantor's original "three principles of ordinal generation". (Here, an ordinal is just a certain type of a set.)
The three principles exploit the notion of successor, limit, and supremum. Rather than get bogged down in technical details I will appeal to your intuition here. When we apply any one of these principles to a finite collection of ordinals (numbers) - e.g., {0} or {0,1} - we generate one new ordinal (number). Adding one to a finite number yields a finite number and therefore the three principles of ordinal generation do not allow us to escape the finite domain. At this stage, we cannot take the supremum of all finite numbers to obtain an infinite set since we have not yet generated all finite numbers.
The only way to formally obtain an infinite set is to introduce an axiom that asserts the existence of an infinite set - the so-called Axiom of Infinity.
Once we have asserted the existence of an infinite set, we can continue to apply the three principles to generate new infinite sets which are mathematically distinct from that set identified by the axiom of infinity.
You then say "As per my understanding, infinity is fully formed, perfectly symmetrical". This is not entirely correct. Certainly those sets formed by the three principles are fully formed in the sense of being well-defined. However, in the case of infinite sets, these sets are not symmetrical since, for example, there is a smallest whole number but no greatest whole number. Other infinite sets may have a least member and a greatest member, but "in between" there are subsets with no greatest member.
It is also important to understand that our intuition of the infinite extends beyond those well-formed sets generated by the three principles to what Cantor called the absolute infinite. These collections are not well-formed. These are inconsistent infinities such as the collection of all well-formed sets. The inconsistency of this collection is witnessed by Russell's famous paradox.
Finally, you say "So that would imply that infinity is one as a concept and there are no different classes of infinity, and we tend to classify infinities for the sake of simplification in mathematics." This is not correct as a statement about mathematical infinity. As we have already noted, once we have asserted the existence of an infinite set we can continue to apply the three principles of ordinal generation to generate new sets which are mathematically distinct.