Emotional reactions to things are an automated response to your own underlying values. The fact that you experience a feeling of guilt from your rejection of these beliefs therefore suggests that you have not fully abandoned them, or at least, you have not fully abandoned associated beliefs that underpin them. If you are taking a course of action that is contrary to a behavioural standard that you reject as irrational, there should be no guilt. If you experience guilt, this means that you have not fully integrated your rejection of the behavioural standard in question --- there is still some underlying acceptance of the standard which is causing you to feel bad when you break it, or some associated normative rule that is inducing guilt.
In my experience, the solution to this kind of problem is to inquire more deeply into the source of your emotional reaction (i.e., engage in introspection) and identify why you feel guilt at breaching a behavioural standard you reject. Once you have identified this, you will be able to determine if the source of your guilt is rational, and then either reject further irrational beliefs (removing the guilt), or conclude that your initial feeling of guilt was the correct response and alter your beliefs accordingly.
Since you have not specified the beliefs at issue in your struggle, it is hard to give specific advice on the path of proper introspective inquiry. However, reading between the lines a bit, I presume you are talking about adherence to some social convention which you regard as irrationally restrictive, but which is regarded as valuable and desirable in "the culture". If this is the case then it is possible your guilt comes from a sense that you should be obeying the dominant values of your culture, and should not put yourself at odds with the majority. If that is the case, then there may be philosophical questions other than the specific behavioural issue that you need to examine.
You might find the following quote by Rand (1964) (emphasis added) to be useful in discussion of the interface of emotions and values:
An emotion is an automatic response, an automatic effect of man’s value premises. An effect, not a cause. There is no necessary clash, no dichotomy between man’s reason and his emotions—provided he observes their proper relationship. A rational man knows—or makes it a point to discover—the source of his emotions, the basic premises from which they come; if his premises are wrong, he corrects them. He never acts on emotions for which he cannot account, the meaning of which he does not understand. In appraising a situation, he knows why he reacts as he does and whether he is right. He has no inner conflicts, his mind and his emotions are integrated, his consciousness is in perfect harmony. His emotions are not his enemies, they are his means of enjoying life. But they are not his guide; the guide is his mind. This relationship cannot be reversed, however. If a man takes his emotions as the cause and his mind as their passive effect, if he is guided by his emotions and uses his mind only to rationalize or justify them somehow—then he is acting immorally, he is condemning himself to misery, failure, defeat, and he will achieve nothing but destruction—his own and that of others.