There is no difficulty in assuming that we don't actually know anything about the material world. We don't even need to claim to know that we know nothing about it since we may believe it is the case, or indeed believe it is not the case.
It won't make any difference if we can only have beliefs about the material world and it won't make any difference if we can know the material world but are unable to tell the difference between belief and knowledge.
Our possible claims to knowledge can be seen as just beliefs we believe are knowledge. Beliefs we think of as knowledge but are not can be regarded as beliefs free from doubt.
And I don't see why they would have to be any substantial difference in behaviour between people who believe they know and people who believe they believe. I certainly seem myself to keep behaving as if I knew my way around even though I believe that I actually don't.
We can think of human beings as acting according to whatever they happen to believe and we can think of scientific theories as particular beliefs. We don't even need to pretend that scientific theories are knowledge since it is good enough to believe they are true. And that we act accordingly.
There is also no difficulty is ranking our beliefs in terms of the kind of justification we think we have to have them. And we can rank science as our most certain belief about the material world. I certainly do.
And we can also rate our own perception of the material world as more reliable most of the time than anything other people might want to claim they know. And I certainly do.
Is mathematics any different?
Obviously, we think of mathematics as an abstraction, and apparently even most mathematicians do. However, there is also no difficulty if we accept that no mathematician knows any mathematical truth.
As in the case of our beliefs about the material world, it is enough to think of mathematicians as acting according to their beliefs about mathematical truths. Here again, there is no substantial difference between believing you know and believing you believe as long as you act according to your belief, which seems to be what the notion of belief suggests we do anyway.
Knowledge of the material world is similar to the notion of the infinite. It makes no practical difference as long as you don't claim to know that you know. Claiming to know that you know does expose you to the risk of being contradicted by the facts of the matter, or at least by something you may come to believe about them at some point in the future.
Why take that risk? Well, we believe it can be very useful. We believe we can leverage our claim to have something we don't have to gain some substantial material advantages. And we all do it, anyway (I believe).
British mathematician Andrew Wiles may believe he proved Fermat's Last Theorem and maybe he did, and that he did or did not would make a substantial difference, but that he merely believes he did, and that we all believe he did, instead of all believing that we know he did, doesn't make any substantial difference.
Just like when I say "I love you". What matters is not that we should know it to be true, but that we should all believe it to be true, and then not even forever, but just on the moment.
And act on it.
Which explains gracefully why we seem to be here at all to tell the tale.
There is no good reason to assume any substantial difference between our various senses of perception of the material world, on the one hand, and our mental capabilities, such as memory, logical sense, and even feelings and abstract thought, on the other. The latter capabilities are essentially, in all but name, perception of the material world, with the caveat that they are perception of that part of the material we call our body, and concerning our memory and our logic, that part of the material world we call our brain.
Thus, mathematical abstractions behave in every way as do anything in the material world. We perceive them where they are, in this case in our own brain and they appear to us just as characteristically themselves as do the tree in my garden. Differences are just as significant as differences between vision and hearing, for example.
Thus, mathematical abstractions may just be as unknowable as anything else. You know the one you have in mind, but then you can't hold in mind all mathematical abstractions at the same time. And there goes our claim to knowledge down the drain.
Further, since physics is now suggesting a universe which many people like to think of as essentially mathematical--that is, somehow mathematical in nature, whatever that may mean exactly--there is a near perfect analogy between knowledge of the material world and knowledge of the Platonic world of mathematical abstractions.
We are able to believe we know them.