Believing in something and acting upon it are two separate things. Belief is the acceptance of something as true, whether or not it can be proved.
Most rational people generally only act on beliefs that they feel strongly about, either because they're compelled to demonstrate them, or because they feel someone else is opposing them.
A person is likely to have a number of beliefs they hold to be true that they take no discernible action over. Although a belief may subconsciously shape an individual's world-view and influence their behaviour.
Personally, my belief (here we go) is that everyone has the fundamental right to believe in whatever they wish. I accept that as true, although I can't prove it.
However I would be prepared to question anyone's right to act upon their beliefs or impose them on others, and likewise have someone challenge my right to do so.
Is the issue about the right of someone to act in a particular way (according to their belief), or is it about the right (or obligation) of others to curtail or prevent someone from exercising their beliefs (legally or morally.)
It seems to me the rights of an individual are defined not so much by what they can do, but by what they aren't prevented from doing. Theoretically, in a system with no laws, morals or restrictions, everybody would have the right to do anything they pleased.
Are we are enabled by that which doesn't constrain us?
For most people their conscience and a reasonable moral framework limit their behaviour to acceptable norms, this is then further reinforced by the social contract within which they live. Limits imposed internally and externally on how we act, if not necessarily on how we think.
That presupposes that the constraints of society placed against an individual are justified and accepted as being for the general good.
Perhaps then there are two notions - the fundamental right to do (or not do) something and the actual tangible 'right' to attempt something without censure or prohibition. In an oppressive regime for example, the fundamental rights of the individual may be repressed by the state as it exercises the perceived right (or beliefs) of the governors in restricting the freedoms of the governed.
Deciding which rights qualify as fundamental is a thorny issue, particularly when one set of rights comes into conflict with another. What seems like a reasonable belief for one group of people to act upon may not be seen as such by another.
A contentious example (from a Brit): in the U.S. many hold that the 2nd Amendment exercises the right to a belief that individuals can bear and keep arms for self-defence, and to maintain the capability to rise up and overthrow oppressors (subject to interpretation.) This comes into conflict with the right (or fundamental belief) that your children should be safe at school from homicidal maniacs wielding readily obtainable firearms. Such gun-wielding maniacs would also have been exercising their personal beliefs when they carried out their deplorable acts. Deciding which beliefs people have a right to act upon isn't always straightforward.
Another more facetious example: here in the U.K., proselytizing religions enjoy exercising their right to try and convert people to their beliefs, often by going door-to-door. A householder may believe in the right to form their own opinion on faith, they may also believe in the right to privacy in their own home, free from interruption by others trying to impose their beliefs. Thus the householder may exercise their beliefs by ignoring the knock on the door and waiting for the faithful to move on. Were they instead to pour a bucket of paint out of an upstairs window on these callers, then by their actions, their beliefs would bring them into conflict with the laws of the land, which frown upon such activities.
Clearly the right to act upon a belief can be challenged. But does the lack of a restriction on what you can think, qualify as the right to think it? I'd say so, if for no other reason than it is not currently possible to know with certainty what someone is thinking, if they keep those thoughts to themselves.
If it was, then homicidal maniacs would be prohibited from owning firearms, since that would arguably be a reasonable restriction to place on someone, based on potential acts they might be thinking of doing.
But based purely on someone's beliefs - their thoughts - not by actions they haven't (yet) committed, can you decide what rights that person does or doesn't have?
As things currently stand (as far as I am aware) an individual cannot be prosecuted solely for the inner workings of their mind, however controversial, distasteful or contrary to the accepted social contract or code of laws these thoughts and beliefs might be. Providing they don't act upon on them.
"There is nothing either good or bad but thinking makes it so." - William Shakespeare (Hamlet Act II, scene ii.)
I would amend that to say instead ".. acting makes it so." An 'evil' thought is neither good nor bad until it is acted out.
It's the right of someone to act on their belief (as if it were acceptable) that's debatable. One could also argue that the right not to act (or to be forced to act) as a result of a belief is an issue. I'd suggest that we're probably more inclined to support the rights of another not to act against their beliefs, than we are to uphold the fundamental right of someone to act according to their personally accepted truths.