Edit: You state that if a pointer is just a memory ref, than that means all memory can be pointers. 

A similar ask in an ontological point of view is that if matter points to other matter (such as a human pointing to an apple) that means that it is possible that the apple can point back.

I believe you are making a case for panpsychism, the idea that all matter has conciousness, by eluding to the fact that if matter is conciousness (a pointer is memory) than any matter can be conciousness(all memory can be a pointer).

Marko: Everything in the universe has conciousness.

Chris: Nothing in the universe has conciousness.

Edit:
The referendum is important because there is no evasion. There is an ontological argument being made, in my interpretation, which is panpsychism. I guess you can say that people just don't want to admit it or don't realize that they are doing so...