Your question seems simple, but is very complicated indeed!
Instead of giving a complicated answer ;) I want to bring to your attention the problematic nature of an assumption in it, which I will call Principle P. This assumption is important because if Principle P doesn't hold, then your question cannot even get off the ground.
Your questions seems to imply that
a person A who voted for the currently elected official X is responsible (and therefore to blame, culpable, etc.) for the state of political affairs brought on by X. (Principle P)
This seems a reasonable assumption, at first: The act of voting caused the election of X. If we take for granted that there is a link between causation and moral responsibility, then it would seem that A's voting makes him morally responsible for the election of X. Additionally, A is not just morally responsible for the act of voting, but also for the downstream consequences of this act.
There are, however, two problems which renders this assumption worthy of being questioned:
How far does moral responsibility go?
A is not just morally responsible for the act of voting, but also for the downstream consequences of this act. However, there seems to be a limit as to how far this chain of consequences can go: Voter A caused the election of X; X brought on a certain state of political affairs; did A bring on that state of political affairs? Causation may be seen as a transitive relation (if x causes y and y causes z, then x causes z), in which case the answer is yes. However, it is dubious under which conditions moral responsibility can be transitive in the same way: If A is morally responsible for the election of X; and if X is morally responsible for the current state of political affairs; is A morally responsible for the current state of political affairs?
An affirmative answer to this question becomes even more improbable if we take a very basic principle in ethics that A is morally responsible (and therefore potentially morally blameworthy) for an occurrence E only if E is a consequence of some culpable act on A's part. Since the act of voting is in no way a culpable act per se, A cannot be held responsible for any nefarious consequences of this act.
Therefore, Principle P fails.
Does partial causal contribution lead to moral responsibility?
In a setting such as mass elections, where group agency causes the election of X, what is the causal role of a single vote? Somewhere between very little and none. If we agree that moral responsibility diminish as the degree of causal contribution diminishes, then the moral responsibility is very little to none - even if it isn't easy to precisely quantify the causal contribution of each of the voters to the occurrence of the election's outcome.
Still, we could envision an electoral tie. In this context a vote can be "more" causally efficacious then others. This is called the vote's "pivotal" (i.e. effective) role in the election's result. However, it can be shown that the probability that A's vote is effective and affects the election's outcome is practically non-existent (e.g. minuscule wrt to A's probability of winning a lottery!). Again, A's moral responsibility for the election's outcome is almost non-existent.
Therefore, Principle P fails.
Conclusion
If Principle P doesn't hold, i.e. if even voter A who voted for X is not morally responsible for the state of political affairs brought on by X, then it is not really conceivable how a voter B, who didn't vote for X, might be morally responsible for the state of political affairs brought on by X.