I think you can argue that mathematics can be sublime based on how mathematics is able to invoke emotions and inspire even those who aren't directly involved with solving the equations.
To give a few examples, I would argue that Conrad'sConway's game of life as a mathematical topic is able to demonstrate how complex patterns can develop out of simple rules.
Similarly the Mandelbrot Set shows us an abstract and beautiful pattern resulting from the continous application of a simple rule, which I at least for my own part find inspiring similarly to a work of art in it's reflection on how complex patterns in the world around me may arise from simple things.
In the realm of number theory, primes and pi have inspired people for centuries, not just because of their mathematical properties but also by virtue of an unpredictable behaviour (pi is irrational, primes are hard to predict) emerging from a deterministic system. Classically this unpredictable behaviour was so frightening to the Pythagoreans that they supposedly resorted to drowing Hippasus when he proved the irrationality of sqrt(2) although this is probably not entirely based on reality.
Whether these examples are enought to justify mathematics as sublime is up to you to decide, but I think it's fair to conclude that mathemathics aremathematics is able to both inspire, frighten, and amaze.