The traditional role of a taboo is prohibition of an action, not of discussion, but the two are often mixed when the term is used loosely, see e.g. Gao's study of English "taboo" words. Taboos against homicide or incest had obvious biological/social benefits. Volume 3 of a classical comparative study of mythology and religion, Frazer's Golden Bough, is called Taboo and the Perils of the Soul, and freely available online. Freud's Totem and Taboo is now considered discredited by antropologists, although some modern authors see residual value in it.
Ellis authored an essay The Function of Taboos, where he argues that taboos play an adaptive evolutionary role and are found even in animal communities:
"Among wild birds in a special phase of bird-existence it is taboo to remain close to humans. That taboo is strictly analogous to human taboos; it is an adopted custom. It is not found everywhere among birds. When men first visit Virgin islands of the southern seas there are birds who do not regard human beings as taboo. The taboo is introduced later when human beings have become destructive to the bird society. It is, of course, completely unnecessary to be aware of the reason for the taboo, and if birds ever acquired speculative minds they would invent reasons. That is, as we know, exactly what human societies do.
[...] Life is livable because we know that wherever we go most of the people we meet will be restrained in their actions towards us by an almost instinctive network of taboos. We know that they will allow us the same or nearly the same degree of freedom and privilege that they claim for themselves. The individual in whom the taboos necessary for such organization are not either automatic or self-imposed is an anti-social individual, and his elimination would be for our benefit... Old taboos can only be replaced by new taboos... If they are thus to become of the nature of taboos they must be few in number, indisputable in value, and so urgent that they are felt to be on the way to become instinctive. Sex taboos are at the centre of this process...
Even the "should be open-minded and free to discuss all topics" view, common in Western liberalism, is far from universally accepted. In some cultures it is criticized for leading to social and political instability, for example. And even Western liberalism has taboos of its own, against racism, sexism, etc. Despite the opposing backgrounds both seem to fall under Ellis's description of the function of taboos as maintaining social cohesion. See also Schliesser's post Žižek and Kant: on Philosophical Taboos.