If you replace your memory, you change your person.
A simple analogue: If you change the harddisk of your notebook and insert the harddisk of a friend's notebook, do you consider the equipment any longer your tool? Or do you feel completely bewildered, unoriented and unable to continue your work?
Identical twins start with nearly the same memory. But they develop into different persons due to the different experiences stored in their memory.
A different kind of questions is to ask which change of your mental capabilities does change your personality. One knows that severe damages of the frontal cortex may change the personality of the person. A whole range of possibilities exists: The person still considers himself the same person but with some capabilities restricted. At the other end, the person does not know any longer who he is.
From Stanford Encyclopedia of Philosophy, keyword Personal identity:
What is it to be a person? [...] The most common answer is that to be a person at a time is to have certain special mental properties then.
I take this definition as basis for my answer. All mental properties are embodied in the neuronal connections and theír weights in the neural nets of the brain. They make up the memory, taken in a broad sense.