This goes along with Feynman's criticism of vocabulary as a part of scientific knowledge, especially at a basic level. His story here illustrates why he thinks words out of context are a waste of time. What really happens, when you make a description, is that you discover what you can predict and understand based on a concept or measure, and that power sells the definition, even if it does not really fit any more ordinary definition you might propose.
We are misled, when a physics class starts with definitions, into thinking those are important basic concepts. Really, they are convenient things to slap labels on, that happen to have explanatory power. Consider Newton's definition of 'work', for instance (the movement of a mass over a distance). It is an important measure. It sort of needs a name, and for a very backward sort of perspective, that is sort of an appropriate name. But is that work? The difficulty of most of the things we consider work (and thus how much work is done by accomplishing them) is not directly related to how much mass moves how far...
So while it is a necessary motivation to look at a goal and start guessing measures that might capture the concepts implicit in the goal, it is a bad idea to become attached to the motivation, because the thing you define might grow and change and end up not really fitting the name or its more basic conceptual frame.
Modern physics is even more disconnected from common-sense definitions. We end up, for instance defining mass as a form of energy. So if you cling to the notion that mass is what makes things fall, you are going to be very confused. If you think mass is what you measure on a scale, you are wrong, we can go to space. If you think of mass as the source of gravity, you are going to have trouble with general relativity, where gravity and acceleration cannot be distinguished.
Advancing theory redefines your terms, as we have continued to modify our perspective on mass. So Poincare's approach is the one that prevails. Science has to be ready to give up common-sense definitions or metaphysical motivations, and go with the flow of theoretical modifications wherever that takes us.
He is mildly overstating his case. The metaphysical concept worked as motivation.
Without the notion of force from common-sense experience, we would not have looked where we found what we needed. But to think of that as the proper definition of force for all time would eventually get in our way.