I have read the authoritative books in favor of freedom of speech (and in favor of freedom in the larger sense), but for the sake of learning, I wanted to know what were the authoritative books and arguments against unlimited freedom of speech. I can't seem to find a list for that.
...the authoritative books against [freedom of speech]
Two passages from Orwell's Nineteen Eighty-Four : (1) O'Brien's conversation with Winston Smith while interrogating Smith in prison; (2) The Book, which explains the social system of Oceania, and which Smith reads after joining The Brotherhood.
Perhaps you need to clarify what you intend 'freedom of speech' to cover and include. Below I take it that you are concerned with freedom of speech in the sense of the expression of ideas, principles, beliefs. One might call this a political construction of 'free speech'. However, do the many socially imposed constraints on certain words and expressions that are widely deemed to be unseemly also fall within the scope of freedom of speech ? There are many words, not at all expressive of ideas, principles or beliefs, which are unusable because they violate social mores. There are also questions I cannot ask people (not that I'm inclined to) because they involve a violation of privacy. Involved here is no less a constraint on freedom of speech than my being forbidden to express (say) racist views.
That's just a clarificatory point. Down now to what I think is your immediate interest.
Sir James Fitzjames Stephen's 'Liberty, Equality, Fraternity' (1874) is a wholescale attack on John Stuart Mill's 'On Liberty' (1859). Mill defends freedom of speech; Stephen takes an opposing view.
Among more recent texts see 'Speech and Harm: Controversies Over Free Speech' (Oxford, 2010). For a feminist perspective see Eric Hoffman, 'Feminism, Pornography, and Law', University of Pennsylvania Law Review, Vol. 133, No. 2 (Jan., 1985), pp. 497-534. James Weinstein, 'Hate Speech, Pornography, And Radical Attacks On Free Speech Doctrine' (1999) is worth reading.
On pornography (which may not be your particular concern) Philip Jenkins, 'Beyond Tolerance: Child Pornography on the Internet' (2003) is useful.
I would think that this argument will be very ill received because it criticizes revisit principles of religion.
In the big Abrahamic religions a set of rules have been written down, in some cases to prevent the spread of diseases, the prohibition against pork in Judaism and Islam, in other cases to control a population, the burka or niqab i Islam.
Eventhough Jesus sought to get rid of these rules the church later created Hell and imposed decency and good manners.
In all religions springing from Abraham a certain criticism of the free speech have later been developed. An example of this is the Muhammed drawing by Kurt Westergaard. Here islamic leading figures required an apology from the state of Denmark, the newspaper who printed them and the cartoonist himself, by referring to scriptures not allowing it, written in religious books.
There are many examples in Israel where freedom of speech have been under attack by the Orthodox jews.
A religious censorship has resurrected. Since religious books in many countries have strong ties with state policy, Iran and in some cases the US, I would see parts of the bible, the quran and other religious texts as being Authoritative.