Your moral intuition seems to be close to that of Immanuel Kant:
IF I (morally) ought to do something THEN this should have an impact on my faculty of desire (motivation).
The thing you are missing is close to the criticism of Kant developed by Hegel: Principles alone do nothing. They are purely formal. Any kind of bindingness or motivational form can only emerge in your active application of a principle, i.e. in the active determination/judgement that I actually ought to do this because the principle applies to the particular situation at hand (aside: Hegel's criticism is misguided since that was one major aspect of Kant's practical philosophy).
In other words: Your "fact" is about the universality of certain guiding principles and values, not about the effectiveness of them actually motivating people to act morally.
Hence, there is a difference between
- It is a fact that X is a moral principle across cultural differences and
- All persons are bound by X in the morally relevant situations
The simple reason is that as long as they do not judge the principle to apply to their current situation as morally relevant, they will not feel compelled by it. That not all people are moral is actually not speaking against objective principles but against people recognizing these principles as morally relevant in situations that actually are morally relevant (and deciding to act accordingly).
In fact, the only compelling aspect of principles is the consciousness of an ought (see Kant's Fact of Reason in his Critique of Practical Reason) - which may follow from or be expressed in principles or "moral laws", but is not identical with them - it needs this recognition per judgement. Kant argued for the objectivity/universality of this recognition of morality, but the universality of morality that is based on non-empirical principles is highly controversial (see early criticisms of Kant by Sidgwick and Schopenhauer). And even if you recognise this ought as morally binding, the very idea of "ought" and responsibility implies that you can choose to do otherwise.
Mind, this all is in our moral language (including the idea of a free will). It should not be misunderstood as an assertion regarding the ontological status of freedom or values. Indeed, determining the "absolute" or "real" ontological status of any object of thought (beyond cultural/interpersonal agreement like e.g. "scientific consensus") may simply be beyond us (as Kant held).