Did Aristotle mean by 'Mean' exactly the middle between two extremes, or just some "sweet spot" that need not be the middle at all?
Short answer: it's not the exact middle of two extremes.
Intuitively, this seems implausible because one of the two extremes so balanced might unnecessarily weigh the notion of virtue / excellence / arete) one way or the other. Moreover, this would also seem to have the counter-intuitive result that the notion of virtue / excellence / arete would change depending on the circumstances under which it was considered.
Alright, enough intuition. Here's some textual evidence. First, in Book II of the Nicomachean Ethics, Aristotle makes the following observation:
moral qualities are so constituted as to be destroyed by excess and by deficiency
Aristotle, Nicomachean Ethics, 1104a
Having made this observation, he goes on to define virtue / excellence / arete as follows:
Virtue then is a settled disposition of the mind determining the choice of actions and emotions, consisting essentially in the observance of the mean relative to us, this being determined by principle, that is, as the prudent man would determine it.
And it is a mean state between two vices, one of excess and one of defect.
Aristotle, Nicomachean Ethics, 1106b-1107a
This then is the mean: a point that lies between the two extremes of excess and deficiency, so as to be what the "prudent man" would choose.
But there are some caveats to be observed about this notion of mean. First of all, there is not always a mean to be found:
Not every action or emotion however admits of the observance of a due mean. Indeed the very names of some directly imply evil, for instance malice, shamelessness, envy, and, of actions, adultery, theft, murder. All these and similar actions and feelings are blamed as being bad in themselves; it is not the excess or deficiency of them that we blame. It is impossible therefore ever to go right in regard to them—one must always be wrong; nor does right or wrong in their case depend on the circumstances, for instance, whether one commits adultery with the right woman, at the right time, and in the right manner; the mere commission of any of them is wrong.
Aristotle, Nicomachean Ethics, 1107a
And so, certain actions are to be altogether avoided, despite virtue being defined as a mean. But, Aristotle goes on to point out that being virtuous is hard, and all the more so because:
of the two extremes [i.e., excess or defect,] one is a more serious error than the other
Aristotle, Nichomachean Ethics, 1109a
And so the mean will tend to be a little more to one side than to the other. Moreover, as a practical principle:
one should lean sometimes to the side of excess and sometimes to that of deficiency, since this is the easiest way of hitting the mean and the right course.
Aristotle, Nichomachean Ethics, 1109b
Depending on which way is less opposed to the middle, i.e., which extreme is less bad than the other. And this is why Aristotle's notion of virtue / excellence / arete is sometimes referred to not just as the mean, but as the mean of the mean.
All that said, the "sweet spot" as you put it will be somewhere in the middle, just not necessarily the exactly, mathematical middle.