As Conifold points out, there are lots of weaknesses in Descartes. You are just focussed on the wrong points as weaknesses.
All told, from late modern (or later) perspectives, true atomism is a non-starter in both domains.
Logical atomism failed, and we ended up with "the other" Wittgenstein, and later deSassure, Lacan and other back-and-forth/round-and-round definitions of language that involve interacting constructions, instead of atomic bases. An 'elementary' idea depends more strongly upon usage and intention the more basic it is. And the usage and intention, of course, depend upon agreement between the definitions of elementary ideas...
And someone one hundred years ago would have proposed we have reached physical atomism, as you do, only with different basic particles. They were wrong, then. So why would they be right now?
Strings themselves are not atoms, because they do not keep themselves together as unified wholes: they split and join, and are not integral multiples of some minimal element.
But more in keeping with the other realm, folks in string theory are already looking at a string as something whose outside is also an inside (see Greene's The Elegant Universe and the discussion of the Planck length) so at root physics may have the same problem as language -- there being two different perspectives, each subdivisible, where larger in one is smaller in the other.
In both cases, you can't choose atoms because when stuff gets small enough, it starts getting bigger. So as continue subdividing, you are eventually multiplying. There is a 'most definite' place, a smallest piece, but those bits are not self-contained and fixed in content, or even in the number of them before or after an interaction.
There is a minimal width of a color difference, but that does not mean the number of different colors are fixed, or that two people who agree on a single shade and a minimal difference have the same set of shades. Two strings can cross, and the short ends become one string while the long ends become another, and he middle bit ends up as a loop. Basic elements recombine in different sizes, shapes and numbers.