The quote from Russell is in conflict, not with Utilitarianism, but with some versions of Pragmatism, in particular with William James' Pragmatist Theory of Truth.
Note that Bentham's quote refers to the good and the right in actions; while Russell's quote refers to the holding and to the truth of beliefs. Those are two different areas. A rational person is usually supposed to hold a belief in proportion to the evidence that supports that belief, regardless of any practical consequences that the belief may have. And so, utilitarians are not usually expected to hold the view, that Russell's quote opposes, that a belief should be held according to its consequences.
Some others, however, have indeed held that view about beliefs. One such thinker, with whom Russell personally debated, was William James. In the essay The Will to Believe (1897) James developed an argument, according to which it is allowed and justified, in some cases, to hold a belief even when one doesn't have sufficient epistemic evidence in its favor. The main class of beliefs that James had in mind were religious beliefs. The basic idea of his argument was, that if we have to wait for evidence, we are bound to miss the benefits that some (e.g. religious) beliefs could bring about. In later publications, James developed the Will to Believe argument into a general theory of truth. He came to hold that a "true" belief was just a belief which was beneficial to hold, in the long run. This received the title "the pragmatist theory of truth".
Russell opposed James' theories, on various grounds. One of his arguments was that, even if some (e.g. religious) beliefs were beneficial, what gave them their psychological power was the that the believer believed that they were really (i.e. not in the pragmatist sense) true.
Suppose I say there was such a person as Columbus, everyone will agree that what I say is true. But why is it true? Because of a certain man of flesh and blood who lived 450 years ago—in short, because of the causes of my belief, not because of its effects. With James's definition, it might happen that ‘A exists’ is true although in fact A does not exist. I have always found that the hypothesis of Santa Claus ‘works satisfactorily in the widest sense of the word’; therefore ‘Santa Claus exists’ is true, although Santa Claus does not exist. James says (I repeat): ‘If the hypothesis of God works satisfactorily in the widest sense of the word, it is true.’ This simply omits as unimportant the question whether God really is in His heaven; if He is a useful hypothesis, that is enough. God the Architect of the Cosmos is forgotten; all that is remembered is belief in God, and its effects upon the creatures inhabiting our petty planet. No wonder the Pope condemned the pragmatic defence of religion.
(History of Western Philosophy p.728)