What is the definition of 'belief'
A western belief is, according to the SEP, the
attitude we have, roughly, whenever we take something to be the case or regard it as true
You will notice that this is intentionally vague, especially it does not say anything, or require any measure of "objective" truth (whatever that may be, it is a complicated concept in its own).
Also you will notice it is not related to anything "mystic" by default. You can believe things that are truly physical, everyday things that are not under any doubt at all. E.g. "I believe it may rain later" is fundamentally the same as "I believe the moon is made from cheese" or "I believe there is a afterlife", albeit of course different in the details.
It is a precursor, but not the same, as knowledge (which is often, but not always, thought to be "Justified True Belief", with the conflict mostly again in the word "True", which suggest objective truth, which is complicated).
is it irrational to have one?
It is impossible to not have one; it is a regular feature of our brains, it is in our nature. You can describe it and discuss its implications, but not try to not have it.
It may or may not be irrational to think that a belief is also true in the clear absence of evidence (whatever that is, again! There are many different opinions/interpretations about all these terms).