I would also challenge the framing of the question. If someone treats you like property, or like an object, you would still have moral agency. Right?
Chattel-slavery denied rights to some humans, and made them property, so it definitely can be done, and Plato and Aristotle defended slavery, though of a more moderate kind.
Descartes considered animals to be automatons, that lacked souls, which he understood as about the 'inextensible' world of mind and ideas, separate to language. But most people would challenge this, with for instance 'Not to eat flesh torn from a living animal' going back to the Jewish laws from Noah so older than the Ten Commandments.
Considering others to deserve moral consideration, or to have moral agency, is closely related to whether that person is considered to have personhood. This is a whole topic in philosophy, bearing for instance on abortion, whether or when an unborn child is considered to have personhood. Peter Singer who provided the philosophical grounding for modern animal rights in his book Animal Liberation, makes the case chimpanzees and dolphins deserve consideration of a higher standard of personhood than other animals, because while they cannot be held responsible for their actions quite like a human can they none-the-less have greater capacities for moral reasoning than other animals (eg chimps value fairness) .
Environmental personhood, is a movement to grant special consideration to certain natural features, such as the Ganges river, making them valued in a special way, and given rights.