Your example is related to the so-called "knowability paradox" concerning sentences of the form "p but p is not known", pointed out by Church in 1945. The
Church sentences are not contradictory, but simple argument shows that they can not be known for any p. Indeed, if the Church sentence is known then p is known, but it is also known that p is unknown, which is incoherent. In other words, if there are unknown truths (ones that never become known, if we think temporally), then there are unknowable truths. Church may have been inspired by Moore's paradox, proposed in 1942, featuring sentences like Moore's "I went to the pictures last Tuesday, but I don't believe that I did". Moore's point was different though, that such sentences create a contradiction whenever uttered, because (honest) utterance requires belief.
If one adopts an epistemology under which there are no unknowable truths (like intuitionism), then one must accept that nothing not known can be true (or one can adopt some non-traditional epistemic logic). To put it positively, "if p is true then p is known", the inference you are using is that with p="X is impossible". This is unpopular, but not as crazy as it sounds. The belief in unknowable, or verification-transcendent, truths is a hallmark of realism. Anti-realists (about a particular domain) impose stringent proof-theoretic requirements on knowledge, so that nothing not supplied with a proof is considered true, and anything supplied with a proof will of course be known to be true. On this model of truth if X is impossible then you already know it, and if you do not, then no truth value attaches to the claim. This is why intuitionists and anti-realists reject the law of excluded middle, we can not know that p or not p without knowing which, otherwise we admit unknowable truths. If one adopts such a notion of truth then epistemically possible does imply possible (and so the opposite is false, and hence unprovable), but then there seems to be little point to distinguishing between epistemically possible and just possible.
Wittgenstein held something like such a position concerning mathematics in his intermediate period, according to him "a mathematical proposition is an allusion to a proof". On Shanker's reading, to Wittgenstein unproven conjectures have no truth values because they have no meaning. Here is a commentary from Matthíasson's thesis:
"Wittgenstein abandoned the view that language had one underlying logic or calculus. He now believed, says Shanker, that it consisted in “a complex network of interlocking calculi: autonomous ‘propositional systems’ each of which constitutes a distinct ‘logical’ space”.
[...] The relation of a proof to its proposition is internal and creates the meaning of the mathematical proposition, i.e. the role of proof is not to merely convince its reader of the truth of the proved proposition (which would be an external relation on this picture) but is necessary to establish the very meaning of the proposition being proved — a proof is thus an essential part of the proposition it proves.
[...] This of course immediately raises the following problem: If the meaning of a mathematical proposition is dependent on its proof, a mathematical conjecture changes its meaning when it has been proven. It then follows that a mathematical conjecture can never be proven (since the proposition proven is not the same as the one conjectured). For Shanker, conjectures are strictly speaking meaningless but provide a ‘stimulus’ for the mathematician to come up with a proof, and thereby a new calculus."
