Everything is not perception.  

> Perception comes from the brain.

Does it? If everything is perception and perception comes from the brain, are you saying anything other than perception comes from perception?  

If perception comes from the brain, how would you even know that everything is perception, or, be able to tell if everything were not perception?  

If everything were perception how in the world would you even have the language to pose the conditional question, "what is the physical world if everything is perception?"?

If everything were perception, you would have no means by which to verify the ontological status of the world and confirm any epistemic claims about the world.

For all we know perception *happens* in the brain but it is insufficient to say that the brain *causes* perception without any clarification such that a neuroscientist would confirm. When light or audio waves reflect of a surface and fall upon optic and auditory nerves they "cause perception" as much as the internal neurological mechanisms relay the stimulus through your sensory nervous system.

> By the way, I assume that 'physicality' is a mere perception created by the brain. It simply seems logical to me following the lines of mechanical thinking. Is this a bad assumption?

It's more than just a bad assumption, what you are wrestling with is what Searle calls, "The Bad Argument". Read Searle's "[Perceptual Intentionality][1]" if you are tempted to a denial direct realism.

> The proof then that there two different senses of “aware of” are being used here is that the semantics are different  In the intentionality sense, the subject S has an awareness A of object O implies A is not identical with O; but in the constitutive sense where subject S has an awareness of entity O, A is identical with O, the painful sensation and the awareness are identical  Now let us apply this to the famous Argument from Illusion that we considered earlier  In the sense in which I am aware of an object when I look at the desk, the intention- ality sense, in that sense when I have a hallucination I am not aware of anything  There is nothing there; hence I could not be aware of any- thing  Nonetheless, I am having a conscious visual experience and it is tempting, given the way our language works, to erect a noun phrase to stand for that awareness and make it into the object of the verbs of per- ception  So “aware of”, “conscious of”, are used in two different senses  We feel immediately hesitant to say that one “sees” anything in the hal- lucination case, so we are tempted to put sneer quotes around “sees”  But what is going on, I hope, is obvious and clear  In every case there is an ambiguity in the crucial phrases “aware of” or “conscious of”; because in the intentionality sense in which I am aware of something when I see it, in the case of the hallucination I am not aware of any- thing  I have a conscious experience, but that conscious experience is not itself the object of the experience; it is identical with the experience 

***
> What is the physical world if everything is perception?

If everything is perception, then the physical world is perception.

  [1]: http://www.klemens.sav.sk/fiusav/doc/organon/prilohy/2012/2/9-22.pdf