Utilitarianists judge actions based on their utility value for all people involved. This utility value, or function, is based on mathematics: we are summing numbers.
Mathematics involve the possibility of an infinite number of numbers. Under such a circumstance, we face certain problems, such as what is infinity plus 2? What is infinity minus 5? What is infinity - infinity? And so on. Some of these have answers (infinity + 2 = infinity), and others have no general answer (infinity - infinity = .... ).
Either way, this is a huge problem, and Utilitarianism doesn't seem to be able to cope with it. I'll give examples in both cases (infinity - infinity, and infinity + 2).
Say we are two judge the combination of two actions, one of which has infinite utility and the other which has negative infinite utility. Thus, assuming the actions are independent, the total sum of these two actions is infinity minus infinity .... which is not defined, hence a utilitarian cannot cast moral judgement.
Likewise, say we are two judge the combination of two actions, one of which as infinite utility and the other has utility equal to 2. Again, assuming they are independent, the total utility is infinity + 2 .... but that is still infinity. There, a utilitarian is forced to conclude that the action which has positive utility of 2..... actually is irrelevant. It doesn't matter. Even though it is literally making people happier, since infinity + 2 is still infinity, the value is the same, so nothing is different...