I suppose that not only Aristotle, the Epicureans, and Nietzsche, considered some "pursuits to be better or nobler than other" pursuits - but that most people in full hold of their senses (read: everybody except Bentham) can see that there is more merit in winning a game of chess, or soccer, than in winning a spitting championship or a drinking contest - even if no one can "objectively" explain what this difference is.
As virmaior says, Mills tries to compatibilise Bentham's utilitarianism with common sense (if we can call "common sense" a very 18 century-ish belief in human natural goodness), and he fails.
To my opinion, Mills failure is due to the fact that utilitarianism is utterly incompatible with common sense: we do not live counting unities of "utility", and we could not live doing it: at the time that we would be able to compute the incommensurable possible outcomes of our actions in terms of "utility", the time for our effective action would have passed. For a few eons, I would say. The impossibility of "objectively" measuring something that is "subjective" almost by definition, such as "utility", thence, is a relatively small part of the problems of a philosophical "tendency" that is far more problematic than that.
From a different perspective, I fear that the utilitarian problems with objectivity/subjectivity cannot be addressed without cutting the Gordian knot of utilitarian philosophy with Marx's razor as posited in the Eight Thesis on Feuerbach:
All social life is essentially practical. All mysteries which lead theory to mysticism find their rational solution in human practice and in the comprehension of this practice.
Utilitarianism relies on the belief that not only social life is not essentially practical, but that human life is not essentially social. Thence intellectual exercises in foolishness such as
In a hospital there are four dying men. Each could be saved by a transplant of a different organ, but no donors are available. In the hospital waiting room is a healthy man who, if we killed him, could provide the requisite organ to each dying patient, thereby saving four lives for the price of one. Is it morally right to kill the healthy man and harvest his organs? - Richard Dawkins
Of course, one would answer, if we want people to go to hospitals when they feel sick, Dr. Dawkins' idea is a very bad one. But then we would have to realise how a hospital works in social practice, which is difficult when we do not want to recognise the existence of a social practice that isn't the mere sum of individual, egotistic decisions - or, on the opposite side of error, that isn't a totalising abstraction that negates the individual necessities and contributions that compose the fabric of social reality (everybody should lose their jobs for the greater good of the "economy", people are suffering horribly but this is no problem because the country is on the right track, etc).
To that extent, I fear that the disjunction between "objective" and "subjective" is part of the problem. Most social interactions do not belong to either category; they are firmly "intersubjective" and cannot be understood under the usual binary. Similarly, the discussion on the nobility or "value" of individual behaviour cannot be framed under the category of "objectivity" without resource to some hypostasised "common good", or left on the grounds of "subjective" individualism without negating the very ground on which ethics stands, which is social life. The "objective" reality of social practice only exists as it is daily and mundanely built by our social relations, and the question isn't how to get rid of those social relations, nor how to satisfy those social relations at the expense of our individual sacrifices, but how to build social relations that enhance the life of individuals.
That would, in a simplistic reduction, be the distinction between higher and lower "pursuits": those that help building social relations conducive to happy individual lives are superior to those that help destroying social relations, or help building oppressive social relations.