I will present a possible explanation for why the fiercest defenders of natural sciences often ended up with the most idealistic metaphysics in the history of the philosophy of science:
Science is incomplete without a metaphysical background, i.e. methodological and ontological reflections on its very possibility. Hellmuth Plessner delivers a nice argument within chapter two of his The Levels of the Organic and the Human [Die Stufen des Organischen und der Mensch; translation forthcoming] for the link between natural sciences and idealistic metaphysics (the only one I am aware of atm, tbh).
According to Plessner, the problem is that science (i.e. natural science, as opposed to humanities) has to methodologically rely on material reductionism, i.e. all that is scientifically accessible is that which is measurable, i.e. the physical world, and the world ends up looking like only the measurable [physical] stuff was actually existing (analoguous: If all you have as a tool is a hammer, stuff starts to look a nail).
In that mode of thought, there is a hidden Cartesian dualism underlying, a division of Being into res extensa (material world) and res cogitans (mind).
As there is no explicable transition possible between these dichotomic spheres, we end up with two basic possibilities for metaphysical foundations (without giving up the Cartesian divide and the standpoint of experience [Erfahrungsstellung] of natural sciences): Reduction of either the mental into the physical (abstracting from any qualia) or the physical into the mental (which includes both qualia and other data).
To paraphrase what this means as a matter of metaphysical consequences: Either we end up baffled, not being able to explain mental phenomena like "colour", "pain", or "intuitive understanding of other humans" at all (except as handwaving them as "epiphenomena" of some neuronal activity), or we end up with (absolute) Idealism, since the mental sphere is undoubtably existent and "nearer" to our consciousness than the outer world (the main line of argument of Descartes himself) and able to include all of this.
TL;DR: Given the Cartesian dualism that is underlying scientific methodology, reducing the physical sphere into the mental, i.e. Idealism, is effectively better capable of explaining why the two spheres (physical/mental) should converge and how they relate to each other. Hence, there is a "natural" tendency underlying all scientific endeavour to lead towards idealistic metaphysics.
Note: This is a rather rough sketch of the whole argument, but the parts that are important for the question are included.
Personal comment: The same problem occurs between data and theory in science, and lead up to Quine et al.