Wittgenstein discusses at length the meaning of unproved conjectures, and of the status of their truth value, during his middle period in Philosophical Grammar and Philosophical Remarks, and during the late period in Lectures on the Foundations of Mathematics and Remarks on the Foundations of Mathematics. Although much in his views changes, he basically comes to the conclusion presented in the OP: the talk of truth/falsity of (typical) unproven conjectures lacks sense. At least, unless there is an algorithm that can be used to "work them out", a "decision procedure". Essentially, Wittgenstein rejects the platonist analogy between mathematics and physics, where we pretend to have a God's eye for the infinite and interpret unproven conjectures "by analogy" to hypotheses about physical reality, and mathematical research "by analogy", often invoked, to geographical discovery of new landscapes.
Panjvani calls this the "no-conjecture thesis", and discusses it at length in Wittgenstein and Strong Mathematical Verificationism, as well as how it evolves from the middle to the late period. For the late view see Empirical Regularities in Wittgenstein's Philosophy of Mathematics by Steiner and Dawson's dissertation Leaving Mathematics As It Is: Wittgenstein’s Later Philosophy of Mathematics.
According to Wittgenstein, any "creative" proof (not a routine calculation whose steps are known prior to being actually carried out) changes the mathematical system it is given in, establishes new conceptual connections and hence alters its rules of use. And, in contrast to physics, mathematical sentences are "grammatical", rule-making, not empirical, so it is the system that gives sense to them. Therefore, saying that the conjecture is true or false prior to such a proof being given appeals to a yet non-existent sense, a nonsense.
The sentences like the Goldbach conjecture, where there is a decision procedure in case of falsity only, are special. But even with them the meaning of their truth, as established by a proof, will be different from what truth/falsity could mean before it. And in the case of the twin prime conjecture it is altogether lacking in advance. Here is from Wittgenstein's discussion of the Goldbach conjecture from Philosophical Grammar, ch. 5:
"Only the so-called proof establishes any connection between the hypothesis and the
primes as such. And that is shown by the fact that... until then the hypothesis can be
construed as one belonging purely to physics. - On the other hand when we have
supplied a proof, it doesn't prove what was conjectured at all, since I can't conjecture
to infinity. I can only conjecture what can be confirmed, but experience can only confirm
a finite number of conjectures, and you can't conjecture the proof until you've
got it, and not then either.
[...] Suppose someone was investigating even numbers to see if they confirmed
Goldbach's conjecture. Suppose he expressed the conjecture - and it can be expressed
- that if he continued with this investigation, he would never meet a counter-example
as long as he lived. If a proof of the theorem is then discovered, will it also be a proof
of the man's conjecture? How is that possible?"
In other words, what "the man" had in mind when he conjectured that even numbers are sums of two primes was some sort of physical prediction requiring infinitely many checks. And confirming such a thing is not humanly possible. Even if a "creative" proof "by principle" is then discovered it will not be "confirming" this at all. It will instead establish a new connection between the conjecture and the primes, so it will prove something else, something without a sense at the time of making the conjecture.
"a mathematical proof incorporates the mathematical proposition into a new calculus,
and alters its position in mathematics. The proposition with its proof doesn't belong to
the same category as the proposition without the proof. [...] So if I want to raise a question which won't depend on the truth of the proposition, I have to speak of checking its truth, not of proving or disproving it. The method of checking the truth corresponds to the sense of a mathematical proposition. If it's impossible to speak of such a check, then the analogy between 'mathematical proposition' and other things we call propositions collapses.".
Things are even worse with the twin prime conjecture where no sense of "checking" can be attached in advance. Even if it is false, how would we check (non-"creatively") that the last pair of twins is the last?
"How strange it would be if a geographical expedition were uncertain whether it had a
goal, and so whether it had any route whatsoever. We can't imagine such a thing, it's
nonsense. But this is precisely what it is like in a mathematical expedition. And so
perhaps it is a good idea to drop the comparison altogether."
I will not go into the changes that Wittgenstein's thought underwent in the late period, and how it altered his view of unproven conjectures. In a sentence, he came to associate the sense of mathematical propositions not with a rigid calculus and decision procedures but rather with looser "language games", mathematical practices. But, suffice it to say, the conclusion about the lack or change of meaning of unproven conjectures after a proof remained in place. As well as the anti-platonist idea that their truth is not "determined" in advance by "mathematical facts". Except, perhaps, in an empirical sense of what future mathematicians are likely to accept for compelling, but ultimately empirical and/or pragmatic reasons. Here is again on Goldbach from Lectures on the Foundations of Mathematics (XIV):
"Suppose someone said, "What you, Wittgenstein, say comes
to saying we could also extend arithmetic in such a way as to prove
I this is not so, or to make it a primitive proposition." I'd say:
certainly. Because of course you haven't yet made this extension. The
road is not yet actually built. You could if you wished assume it
isn't so. You would get into an awful mess.
[...] If we adopt the idea that you can continue the road either in this way or in that way (Goldbach's theorem true or not true)
- then a hunch that it will be proved true is a hunch that people
will find it the only way of proceeding. Though before anyone has
found a proof we could say, "If someone has found a proof we
have a perfect right not to acknowledge it"."