Pierce appears to harking back to and widening Plato's theory of Naturalism in language. According to this theory, language grew organically from the sounds we could hear around us. This should be contrasted with Conventionalism which states there is no neccesary such link and the reference between sound and meaning arevpurely conventional. On the face of it, Conventionalism appears to be obviously correct. What link is there between the word 'cat' and a cat itself? Or the word 'chair' and a chair itself. We might as well call a cat a brilly and a chair a witke. This would be strongly affirmed by a logical view of language. But language is not logic, nor grew out of logic, it is natural to us and we live in nature. Even our own artifice - cities and the like - which we set apart from nature, calling it artificial, is actually natural in the same way a birds nest is natural.
But what evidence do we have for naturalism? One clue is onomatopaic words, words that sound like what they refer to. The 'woof' of a dogs bark. The 'whoosh' of a plane going by. But of course the discovery of human speech is lost to time. Sound leaves no traces that we can investigate. But we do have traces of another form of human speech - writing. And there we see writing began naturalistically. Look at Egyptian heiroglyphics and how our alphabet developed from there. It began naturalistically and this is strong circumstslantial evidence that speech began in the same way.
The root of science and the first science to be discovered was number and geometry. Although we are taught our multiplication tables and our Euclidean geometry, number and geometry is instinctive with us. We see the difference between one apple and two apples and we can see that one apple plus two apples make three apples. We distinguish the straight line and the circle from all other curves. And likewise the notion of parallel and perpendicular. To understand this is not to actually learn, but as Plato pointed out, to recollect, as we already know them.
This is the basics of science today: the geometry of physics today is based upon the notion of a connection on a manifold and this is geometrically a splitting of the tangent bundle on a manifold into a horizontal and vertical bundle - and these two are - imaginatively speaking - perpendicular.