The zero on the Celsius scale is still an amount of heat. The zero on the Kelvin scale is absolute zero of heat. It's a useful technical linguistic distinction. Maths doesn't give us privileged access to the world, it depends for it's use in physics on us having correct abstractions, correct methods to calculate, and appropriate understanding of terms and definitions. Personally I think the fervor people get in over zero and infinity (and imaginary numbers) is hype, and we need a humbler understanding of the role of mathematics.
The earliest use of zero was by Ancient Egyptians, for a reference level to build above and below. And then it transferred into accounting.
Indian mathematics introduced the place value system, in which zero is a place-holder. They used the word sunya, as used by Buddhists in sunyata, emptiness of essence. In that sense it was linked to where things arise, and was able to take on a deeper meaning and than for the Ancient Greeks. In a sense this is still relative, emptiness as original nature, identity-less or essence-less being alone, awareness without possession.
The Unruh Effect is an interesting example from physics of the temperature of 'empty space' depending on the observer.
Zero is relative, because it's a concept, not something in the world. The concept derives from comparing something, to it's absence. It is always relative. And something that's zero on one scale may not be on another (eg zero Kelvin is not zero energy), so equating all zeros is basically unjustifiable math-mysticism.
I'd look to Wittgenstein on examples like how the word 'game' derive from modes of life, and get extended by metaphor and analogy, but remain fuzzy ('know it when we see it'), and there are typically sets of 'family resemblances' for their use in different contexts. Zero is like that, even it's relativeness is relative.
