Aim for sophisticated disagreement, not necessarily truth
I always wanted to read and I like reading in general, but I avoided reading many topics like history, philosophy, etc. because I think they are greatly influenced by opinions, beliefs, religion, etc. and I cannot be sure of their validity or accuracy.
As a teaser, I quote from David J. Chalmers's Why Isn’t There More Progress in Philosophy?:
Advocates of a view learn what extra commitments they need to take on to avoid the arguments. Bad versions of a view are rejected and sophisticated versions are developed in their place. This leads to a sort of negative progress where areas of philosophical space are eliminated, but only in small fragments at a time. It is rare for a major general view (materialism or dualism, compatibilism or incompatibilism, utilitarianism or deontology) to be eliminated in this way. Instead, there are large surviving fragments involving the views needed to avoid the arguments [...]. The same sort of elimination, fragmentation, and refinement often recurs at these lower levels. The views that survive yield a sort of fractal structure to philosophical space, akin to the Mandelbrot set with its intricate complexities at all levels, but in which large regions of space are rarely eliminated entirely.
So while it shows that philosophy is much influenced by opinion or "judgement calls", it's not that arguments are completely powerless. We get at least this sort of progress: sophisticated disagreement (as Chalmers calls it).
The idea that in every one of these topics there is an opinion and an opposite opinion, and to reach the truth I had to dig up in 10,000 books, made me hate to read any of them.
You very likely won't reach the truth in all philosophical questions that you tackle, except through some special grace or luck. What you aim for is sophisticated disagreement.
And for this you don't have to read 10,000 books. In theory, one good, non-opinionated introduction about a philosophical field suffices.
The mathematically educated can begin with philosophy of mathematics
Can you please recommend me some books or resources that can introduce me to philosophy in a clear and objective way? How can I overcome my fear of reading topics that are not purely factual or logical?
I guess I'd start with something from philosophy of mathematics, so one learns that not even in mathematics, everything is purely "logical", and there are "different schools" (e.g. intuitionism) and hidden, unexamined assumptions.
There's a bit of an odd book, Philosophical Perspectives on Infinity by Graham Oppy, in the sense that it is mostly philosophy of mathematics but also a bit philosophy of religion. So it bridges those areas of which one is supposed to be very rational and the other quite the opposite.
I'd start with that, in your special case, if you're interested in infinity at all. It should be easily accessible, with your mathematical background knowledge.
Plato
After that, one could read arguably the most important philosopher in the Western tradition, Plato. Maybe the dialogues Theaetetus and Meno (and read the SEP articles for them, they're good to check if you understood everything). Theaetetus is about knowledge, Meno more about mathematical knowledge, and an important Platonic doctrine, the "recollection" of innate knowledge.
René Descartes
I don't know if one should read René Descartes' Meditations soon after that, though in university education one usually begins with this work. Descartes' quest for absolute certainty and the highly questionable method he employs to get out of his radical skepticism had a discouraging effect on me. It reeked of futility. As a beginner, you're kind of left alone with his unnerving skeptical arguments, since you aren't in a position to refute them – they just feel very wrong. So it's easy to take away the message that philosophy is an idle, sophistical exercise.
Still, it's the foundational work of modern philosophy, and especially interesting since Descartes sent the manuscript to luminaries of his time, and published their objections and his replies to them.
So I guess, read the Meditations, but with the correspondence.
C. S. Peirce
Because of the sense of futility Descartes instilled in me, I really liked C. S. Peirce's critique of Cartesian philosophy in Questions Concerning Certain Faculties Claimed for Man and Some Consequences of Four Incapacities.
⚠️ But a warning: while I think they are understandable for a beginner, Peirce uses very few of Kant's technical terms (the "transcendental object"), so one has to look that up somewhere