Yes, this is a common canard, but it basically misunderstands the various notions of idealism that people actually propose.
From the the most basic 'Plato/formality' take on idealism, your will is just one idea among many. And there is no reason to presume other ideas are subject to your will. In fact, within your own mind, you experience counterexamples to this constantly. Most of your thoughts happen to you (as distinct from 'occurring to you'), and you cannot avoid having them. You can only adjust the perspective you take to them. If you cannot make yourself stop thinking about pink elephants at will, or make your mind grasp some concept just by wishing it so, why would you have control over the rest of the universe just because it was made of ideas?
From the 'monadic/process' direction of idealism all we are all complete reflections of the universe, so you may theoretically be in control of what you think, but you are not alone in the universe, and the entirety is a compromise governed by relationships, and coordinated by an overall notion of harmony and respect. This means that by deference to the overall order, you have "pre-consciously" agreed to take in what others put into your mind. Your own ideas represent just a single perspective in this grander scheme, and only by all minds agreeing so could the rules really be any different.
From the 'Hegel/Berkeley' direction of idealism, you are not God, if only because you choose not to be. You are subject to a higher order because you are not ready or willing to be one with God, or God is not yet ready and willing to be One with himself. Until that impasse is resolved, He controls things and you do not.
The laws of physics may only be logic. But it is not necessarily the logic that makes sense to you, but rather what either makes sense of the overall sum of the forms, of the overall sense of what best mediates reality between minds, or the combined will of the Universe to which you are subject.
In particular, for Kant, human understanding is only one form of understanding, and other forms of understanding influence the currency of ideas available, so physics in toto may not appeal perfectly to us. We are predisposed to understand the world, but the world is not shaped only for us. So some parts of that understanding may come more naturally than others.