You are fallaciously assuming your own conclusion
You have, in your assumption here, and in the prior question, and in the answer you accepted, an unquestioned presumption that the material world is causally closed.
Make that assumption, and knowledge of mathematics, or any interaction with abstract objects, is of course inexplicable. You are just in conflict with your own circular presumption.
Matter is not causally closed
However, there is no justification behind this assumption. Matter is not fundamental per our modern physics. It can be created, and destroyed. What IS fundamental -- is not at all clear. We have math equations that seem to describe aspects of our universe that have no matter -- is the MATH fundamental? Some physicists think so. Most don't, but don't have any coherent alternative.
What we have with physics is not a fundamental ontology, we have a pragmatic practice. And that practice presumes several things:
- Mathematics
- Logic relations
- Matter
- Collections of matter into objects
Physics, therefore PRESUMES abstract objects: mathematics, logic relations, material objects (which are matter/abstraction fusions).
Summary
The interaction of abstractions with matter, is intrinsic to physics, and given physics, matter is not causally closed to abstractions.