Frege holds in Der Gedanke that the Thought is the unity of existence because he considers Thought and proposition to be the very same thing, and our cognition of non-propositional objects is secondary - we know through propositions rather than objects or ideas single-handed.
Apparently it was because of the picture theory of meaning that the later Frege rejected the identification of thinking with having ideas "alone", but considering the picture theory he identifies thinking not with ideas but with "grasping a Thought."
The existence of Thoughts is puzzling, since the argument that Frege uses to hold the existence of this Thoughts or propositions is the relation of dependence that they have with the world and how there are no major differences between this relations and the one that "normal" objects have. It seems that Frege expalins by means of this Thought the impossibility of putting into action things we "think" (like when we are unable to explain an idea we have), and successfully "think" when we "grasp" the Thought. I would conclude out of Frege's writing, that the reality of Thought is as real as the reality of objects, which is strange because it give us a seemingly platonic theory of knowledge: the unity of knowledge are abstract entities that exist "beyond" ourselves, "beyond" meaning that while these are dependent of a mind they exist even whey they are not "grasped" by the mind in which they are. But what is the activity of grasping? How does a person 'grasps' a thought?
A Wittgenstenian answer would be that the use of a certain word is correct if there is a community in which a rule is followed and the many users of the same language game can agree with the use of the word, but Frege is not thinking about words, he is thinking about propositions following the idea of the Tractatus which the later Wittgenstein rejects. I don't remember if there is an explanation in the TLP of the meaning of "grasping" or "understanding" a proposition or Thought. The later Frege seems to be between the early and later Wittgenstein, but his theory seems hard to hold unless there is a more or less explicit definition of the meaning of "grasping" a Thought.