Why do we get angry about anything? We could say that many times it is because we have personal issues related to the event or subject in question.
In the case of physicists, that also may or may not apply.
But in my experience, even if the above is the case, it is usually justified. People tend to distort or misinterpret physics claims in order to fit their own beliefs. Specially with modern physics, that has a lot of out-of-the-ordinary consequences, many people take a superficial view and use that as an "explanation" for spiritual stances.
Ken Wilber has an audio conversation entitled "Does Quantum Physics Prove God?" in which he says that it does not, and that those claims come from misunderstandings. The interesting thing is that Wilber himself is a "spiritualist" (whatever that may mean), but he says that that kind of mix of physics and spirituality is both bad physics and bad mysticism.
So when a physicist gets aggro about that, I would consider that it is because so many people has misused physics without really understanding it, simply trying to transfer some of physics knowledge's strength to the "spiritual".