It *is* problematic, but for much more subtle reasons than the one you propose. It is NOT a circular definition--Hume is too cautious and clever to be caught out that easily. It's a heuristic for judgment, similar to *Ockham's Razor*.

However, taken at face value, it doesn't allow for any novel beliefs. If you reject anything that doesn't fit into your existing framework of beliefs, that framework of beliefs will not be capable of any evolution.

Ultimately Hume's beliefs are founded on an axiomatic empiricism, which, due to his other commitments, he has to treat as self-evident. If you share his core beliefs, his arguments are persuasive. But if, for instance, you have a different judgement of what counts as "more probable" then his heuristic will fail to secure an unambiguous result.