The issue you are not considering is that the act of 'observation' carries two distinct modes in language: description and ascription. Description is a passive mode that merely notes and relates a characteristic of the observed: e.g., "John has hands and feet". Ascription is an active mode that imports causal relationships into a characteristic of the observed, and thus implicitly invokes attitudes and responses: e.g., "John has hands and feet that are perfect for a soccer goalkeeper". Importing causal relationships is always dicey. What if John has no interest in being a soccer player? What if John is a horrible goalie despite his hands and feet? Invoking ascriptions creates expectations, and expectations can lead to a host of unfortunate reactions.
The problem with 'observing' race — in the US specifically, and the Western world more generally — is that the concept of race carries a host of group-level ascriptions, many of which are deeply unpleasant and completely irrational. For example, you might not be insulted if I referred to you as 'a white guy', but what if you found out the next day that when I say the phrase 'white guy' I usually mean someone who is stupid, arrogant, dirty, lazy, and ugly, because that's how I think all white people are? You'd be foolish not to think I was racist once you found that out, and you'd be foolish not to acknowledge that I (implicitly or explicitly) insulted you when I said it. There are huge numbers of people in the US who use racial characteristics to imply class differences of intelligence, morality, criminality, beauty, etc. When Pamela Taylor called Michelle Obama "an ape in heels", she was not merely making an ironic comment or stinging critique of the first lady's style choices; she was leveraging half a millennium of stereotypical ascriptions of blacks as bestial sub-humans who can neither be appreciated nor trusted.
It is possible to 'observe' race without invoking any problematic ascriptions. I can say that Cory Booker and Kamala Harris were black politicians running for the office of president, in part because that's how they self-identify. But if you are going to 'observe' race, you had best be cognizant and in control of the ascriptions you are making, because if you are not then (depending on social context) the default cultural ascriptions might be in control of you. You do not have to think of yourself as a racist to be one — there are still plenty of people in the deep south who will try to convince you that 'n**ger' is just the word they use for black people, and they don't mean any harm by it — so make sure you don't fall into that trap.
Just so it's said, please keep in mind that 'race' has no meaningful biological basis. It is a social construct developed as part of a system of oppressive control during the Colonial era, inseparable from the neo-aristocratic ideal that people ought to 'keep to their place' in society, and deserve to be punished if they don't. The moment you invoke that Colonial attitude — wittingly or unwittingly, in any way, shape, or form — you will be guilty of racism.