In Russell's Problems of Philosophy, in refutal Berkeley's argument for idealism -in Russel's words "whatever can be immediately known must be in a mind"- he states:
Berkeley was right in treating the sense-data which constitute our perception of the tree as more or less subjective, in the sense that they depend upon us as much as upon the tree, and would not exist if the tree were not being perceived. But this is an entirely different point from the one by which Berkeley seeks to prove that whatever can be immediately known must be in a mind.
From this I can only conclude that either:
whatever can be immediately known includes something other than sense-data.
sense-data is something not purely mental.
Now if 1 is correct, then the question remains as to what is there besides sense-data that can be immediately known. (Besides "internal" things, i.e: memories, dreams, etc.. We can't take these into account since the point of idealism was to prove that even things existing "outside" are ideas).
But if 2 is correct, then what is the real meaning of sense-data? To explain my current conception of sense-data: I take the "red" sense-data to be the "redness" that I see. and if I were to hear "red" (as could happen to people with Synesthesia) it would no longer be the same thing ("red"), even if it is caused by the same physical thing and there would be a one-to-one correspondence between the "red_"s that I hear and the "red"s that I see.
Hence I take "red" to be something purely personal and mental. So if it is not purely mental (since we're supposing 2) it must have some other meaning as Russell uses it. What is that meaning?