If we are not able to disprove a statement without creating a contradiction like in the previous example, what does it mean for a statement? Does it mean that we can be certain of it? Isn't it just an argument from ignorance to say that we are certain of it?
Let's step back, take off our glasses of objectivity, and ask, what exactly are truth, proof, certainty, and evidence?
This is a high-level philosophy question which depends on your worldview. I'm going to answer from the perspective of a moderately naturalized epistemology, since epistemology is the area in philosophy that addresses these sorts of issues. I'm also going to invoke the language of Stephen Toulmin that he used in his method.
A statement in logic is generally understood as a syntactical expression of a proposition, and it is not the statement per se you are interested in, but the proposition which addresses the semantics and logics of the concepts. When we consider issues like veracity and modality of a proposition, we can generally do within three separate ways: questions of correspondence, coherence, and pragmatism. To complicate matters, the question of evidence (like invoking warrant, backing, and rebuttals) and proof are also complicated, and are very much domain specific.
Also, there are many types of belief where proof and evidence aren't even required, such as revealed religion, in which adherents are certain, that is to say have no doubt (see Hoffer's True Believer) and don't feel the need to reason. Another example are cranks and crackpots in technical fields such as physics, who often claim that fundamental truths are wrong, and they have "proven" it despite the entire field of experts examining and rejecting their conclusions. Even experts in a field may have complicated and sophisticated claims regarding their hypotheses that may disagree, and are certain of it; they often are champions of entire schools of thinkers who argue points back and forth!
If you're looking for satisfaction, look to evolutionary psychology which argues that if you accept evolution as true, then one can see the brain as sort of an engine of inference which works with modality to select from competing propositions to make decisions (studied by axiology). Those decisions have survival value. Is there a lion crouched and hungry in the bushes? Two men may argue, and having the correct answer is essentially determined by who, if anyone, gets eaten. Compare this to two philosophers who argue over what it "means" for there to "exist" a lions in the bush. Both are likely to go home and get a good night's sleep.
What does it "mean" to be certain without proof? It all depends on the context.