This kind of problem is mentioned in a book I have read, but the book did not give a concrete example.
If any such problem existed, this might help me understand human creativity. I think it would also prove that we are more than just a Turing machine.
There is no agreed upon example of this kind.
Let us explore the issues:
First, we cannot even come up with a decent example of a problem not solvable by any formal system. If you state the problem you want to use, I can simply put solving that problem as a basic entity in a formal system, and thus find a formal system to solve it. If you disagree that this defines a formal system, I am going to return the challenge and claim that your problem was not well-defined in the first place.
So, we need to fix the formal system we want to talk about. The question already mentions Turing machines, so let's go for that. There clearly are problems not solvable by a Turing machine, so we can ask whether there are any problems not solvable by a Turing machine, but by a human.
A typical candidate could be something like "writing poetry". Now, I can of course take my favourite poem, and code it into a Turing machine that prints it. Does this Turing machine write poetry?
One might say no, because the poem is hard-coded into the machine. However, looking at Kolmogorov complexity and the results in that area, it becomes clear that "hard-coded into the machine" is not really a well-behaved notion. In particular, I could obfuscate the code so much that finding out that the TM writes that poem essentially requires running it.
If the objection is that someone else has written the poem before, and the TM merely replicates it: With access to a thesaurus, some basic verse rules and a few complex calculations I could create a TM that writes something that looks like a poem, without me or anyone else having had an explicit mental representation of the poem before reading the output. Short of a Chinese Room argument, it becomes difficult to reject this.
So far we have discussed only a single poem. Will creativity show in the long run? So we could ask for a steady stream of "substantially different" poems. Depending on what substantially different means, it might still be easy enough to code a TM for that. On the other hand, it is no longer obvious that a human can do that. Maybe any human has only the capacity [hardcoded? :)] for a certain number of poems. Any proof you could give me that you really can keep writing poetry (issues of mortality aside), I could turn into a TM that keeps writing poetry.
As poetry is just a stand-in for an arbitrary candidate, the arguments above should show that there is no easy way to get people to agree that a certain problem matches your criteria. Moreover, as neither the claim nor its converse are falsifiable, there is no obvious default position on this either.
Every formal system was once the creative construction of an individual or a group of individuals to solve a problem that an existing formal system had not already solved.
The best example I can give is communication. Including both verbal and written, we now have hundreds, if not thousands, of formal communication systems because groups creatively solved the problem of "how do I communicate with another?".
Inventions, including the Turning Machine, were also initially creative until they become integral aspects of a formal system. Consider transportation. Hundreds of examples exist on the different and often very creative or impractical solutions for "how do I get from here to there?".
The formal system is itself a creative solution for "how can I assure others can recreate my creation themselves?". This leads us to "writing poetry". There are formal poetry standards but the first haiku was a creative solution to express someone's thoughts or feelings that got formalized so that others could write a similar poem.
Formal systems are creative constructs of humans. They didn't boot themselves into existence.
The main problem they can't solve is understanding simply because we have no theory of consciousness that works. Turing got around this problem by simply saying imitativeness is enough, and so it has proven to be for most practical purposes.
