A while ago, I asked this question about Borges's philosophical influences, which yielded Berkeley.
Reading the wikipedia page on Berkeley, I can see that as far as philosophers go, his philosophy of subjective idealism indeed exactly what I was looking for:
This theory denies the existence of material substance and instead contends that familiar objects like tables and chairs are only ideas in the minds of perceivers, and as a result cannot exist without being perceived.
But none of the linked articles actually seem to mention a particular text!
I'm not interested in reading any more commentary. I want the real thing. Which text should I read?