There's no one answer to your question, because this is a live debate, and different thinkers have very different moral intuitions about it. Orthodox Catholic theology represents perhaps the position most to the end of one extreme, that the fetus should be treated as a full person from the moment of conception.
At the other end of the spectrum, utilitarian philosopher Peter Singer is perhaps the visible figure who has most fully embraced your exact line of reasoning with all its uncomfortable ramifications --in his point of view, a young child, an intelligent animal, and a mentally disabled adult should all be viewed as essentially equivalent based on their intellectual capacity and ability to live a rich life. He would argue that if it is justified to kill any one of those, it would be justified to kill any of the others given the necessary equivalences.