It depends how one interprets this statement. Offhand, "it's not the time or place for that" is very vague (in the worst case, it could be a non-sequitur).
On the other hand, the "time" argument could legitimately be interpreted e.g. as:
"It is not time to discuss it" ("come back during office hours") or
"At this stage the proposition is not decidable" ("we cant tell because we haven't cracked this differential equation yet/we haven't landed yet a probe on ...").
In the first case, it would have little to do with logic per se, but it would be a social/policy question. The proposition would be decid -able all right, but one doesn't want to go through the process of rationally deciding about it here and now, for some reason. In the second case, it is too early, because there are still a few logical steps to do before getting there.
The moral, is that if the argument "not the time or place" had value (and if both sides were rational), then both sides would likely be able to agree on why it is so (on SE, it could be that it is off-topic).
Of course, in the context you are mentioning (activists, scoffs and scowls, etc.), the hypothesis that all people involved are rational is likely not satisfied.
In any case (especially if you were the only person with your head about you), you could perhaps ask: "Could you elaborate why you consider that this is not the time and place?" and take it from there. This would boil down to the classical: "if we want to argue about it, let us first define our terms".
Here I might get slightly off-topic (as this might pertain to workplace discussions): if you were the person to raise such an objection, you might want to formulate it in a positive way such as: "in order to decide on this, why do we not get together at such and such place and such and such time, since [...explanation...]"?