Thinking about other thoughts requires a reference to something that cannot be present to consciousness. If I make a reference to an apple, I can make that present to my consciousness through sheer imagination of its traits. As studies of the behavior of the deaf without language indicate, this can go really far, and whole communities can live in pantomime for decades. So thought itself does not require language.
But I cannot make a reference to a thought by imagining its qualities, retrieving the thought itself occupies the stream of consciousness, and thinking about it is not possible at the same time in the same way.
This last insight is not a theoretical assumption, or a deduction from the immediate nature of thought. It is an observation based upon the descriptions of thought processes in early literature --http://www.goodreads.com/book/show/402276.The_Discovery_of_the_Mind and the subjective reports of people who acquire or re-acquire language late in life ---http://www.radiolab.org/story/91728-words-that-change-the-world/
The memory of a thought therefore has to be referenced obliquely, and so has to be labeled in some way, either by its relative temporal position, or by outright naming.
So, at least some of the furniture of language needs to be present for thinking about thought to be possible. You need some internal notion similar to the pronoun. The question then is whether that requirement already constitutes language, or not.
For me, it does. At the point where you reference things abstractly, and not by recalling them, you have committed yourself to abstract symbolization. Even if your language has only one word, meaning something like "what came immediately before this in my mind", it has vocabulary, which is used temporally to create narrative structure, and is no longer simply pantomime.