(Welcome to SE Philosophy, and thanks for contributing with a question/response! If you haven't done so, please take a quick moment to take the tour. More specifics can be found in the help center.)
Depends. The question you ask is one regarding statistics and sampling of population, and is a branch of study of it's own. Opinion polling is a popular tool in modern political and policy discourse, and as all things statistical, the devil lies in the details. Whether or not the sample represents the population is subject to the technical arguments of methodology.
The properties and relationships involved here are mathematical in nature, and therefore this is a question intimately related to the philosophical basis for statistics and probability. As I understand your question, you are asking because the majority of the population is much greater than the sample (the population is X, X-Y has not expressed their opinion, and Y that has expressed its opinion is in X such that |X|>>|Y|), is it a logical fallacy to conclude that any conclusion drawn about the population is necessary fallacious because of the bias between those who do and do not express their beliefs.
In normal polling, samples done correctly DO reflect the population as long as certain methodological precepts are obeyed; in other words, it is possible to draw conclusions about populations within certain confidence intervals, and is one of the central focuses of the statistician. Modern society functions demonstrably better by statistics such as these, but where it does go wrong is often related to sampling bias.
It seems you are asking after self-selection bias or participation bias, whereby one's sample is NOT representative on the population either because a group of people have selected themselves into the sample, or some characteristic such as phone ownership (think Dewey defeats Truman) affects the randomness of the sample. Another bias is response bias.
To show that whether or not the sample represents the population is not strictly a question of logic. Rather it is an empirical question. Let's take two possible examples to clarify.
On the one hand, the vocal minority may not align with the views of the silent majority. Perhaps they are motivated and funded by outside special interest groups and are largely ideologues. This happens in contemporary politics frequently on all sides in politics where big money and special interests attempt to achieve their goals through proxies.
On the other, the vocal minority may be a grassroots movement, and the difference between those who express and those who do not express is merely one of time, ability, money, etc. In this case the silent majority might agree with the vocal minority. This isn't atypical in cases like the construction of nuclear power plants or rallies being held by controversial groups like racial supremacists.
The question of whether the former or latter instance is true is a question for empirical research and is the specialty of statisticians. If you have specific questions on statistical methodology, you might find answers in Math SE or more likely in the statistical-centric site Cross Validated.