Yes of course you can, provided your view of the mind is not-non material
Personal identity as a view is not per se tied to any answer the mind-body problem, and there is no logical or obvious conceptual reason why these two views are incompatible, so yes. In fact most metaphysicians who defend the view that you = your body are actually religious in motivation and defend the identity of the body because of the association between the soul and the body in christian sects.
In this case, the memory theorist simply needs to hold a materialist view about the mind. I'm not sure where you have gotten the source on the assertion that materialists believe in the body-identity theory (Animalism).
Edit: I'm not sure why I've been downvoted as it is well-established in the current literature that the primary motivation for Identity being grounded in the body is largely tied to christian belief in ensoulment. This belief for example is grounds for the argumentation used by christians in those who oppose abortion as tying psychological continuity is impossible of very difficult for fetuses who are yet to have meaningful (mental) experiences and thus a kind of memory.