If you have the wit and intelligence to absorb Derrida or Heidegger then you certainly have the wit and intelligence to absorb mathematics or formal logic. One doesn't have to absorb all of the technical apparatus or machinary to become cognisant of its value or to appreciate how it works. Certainly one thing that militates against it is poor exposition; but this charge can be equally put towards learning what Heidegger or Derrida is about - or even Kant.
One does not have to understand how to solve a PDE, a differential equation or the technicalities of Celestial Mechanics to understand the ontology of Physics, or the niceties of Model Theory to understand Godels Theorem or is importance in the notion of truth in its analytic guise.
One might note here that even very good mathematicians are aware of the huge investments of time and effort that it takes to learn mathematics properly (for example Voevodsy - a Field Medallist said exactly this in an interview) and what it means for the continuation of mathematics as a tradition; but then they are in the business of creating and advancing mathematical understanding and that is a very different proposition from philosophy.
Simone Weil, who I would place in the continental tradition, for example had a good understanding of mathematics (her brother was a famous geometer) but in her work there is no mathematics in the mathematical sense; in the sense of rigourous thinking, yes.
High school/college mathematics give very little indication of what mathematics or physics is; it downplays the role of insight and emphasises solving - this is the order of an algorithm or a recipe: One does something in a certain order and one obtains a certain result.