Thanks for all the great suggestions. I find the mathematical orientation answer the best but at some point that just says that you can show that two hands are of the same orientation or different, and still unable to say 'this one is the left one'.

I have convinced myself that, one can tell which is left and which is right by analogy with up and down. We can distinguish up and down sensorially by inner ear changes and proprioception (sense of contraction of particular muscles). It is easier to let the head drop down over the chest than it is to pick it back up; these sensations correspond to the sense of up and down (in space, one doesn't have that sensation, and one has difficulty assigning up and down that way.

As to left and right, one has visual sensors that are on one side and on the the other and those correspond (however one assigns them) to what one can call left or right (anatomically, when we look at the eye, we see that the part of the eye that corresponds to sensing what we call the left of the visual field happens to be on the right side of the eye, but that switch doesn't matter, we can still label what we see (not what we see with) by left or right consistently. (the same reasoning works with proprioceptive sensations in case of blindness).

This is all to say that one can identify left and right mechanically (operationally) by a mapping (simple or complex) to left and right sensors in the body/brain.

In some sense, the 'you just do it' answer comes closest ('the left hand is on the left') to being correct, but there's just no explanation in that simplification.