Your questions belongs to mereology, the (part philosophical, part mathematical) theory of part and whole. We should begin by distinguishing between parthood and proper parthood. Something is a proper part of some other thing iff the first is part of the second, and the two are not identical. Something is a part of something else iff the first thing is a proper part of the second, or both are identical.
Let's call something that has no proper parts a simple, and everything that has proper parts a composite object. Then the view you ascribe to physics is the view that the fundamental objects are simple, while the others, you and me, chairs and tables, are composite.
There are philosophers who think that everything is infinitely divisible (Anaxagoras would be an ancient example). This idea, or stuff that is infinitely divisible, is often called 'gunk'. For gunky objects it is true that every part again has a proper part. (Lewis introduced this term in his book Parts of Classes.) There is also a wiki-article for gunk.
As to the contemporary state of physics, I'd like to point to the efforts of Arntzenius (2008), who argues that a point-free physics can get around some problems. Instead of points (which are indivisible) Arntzenius proposes gunky space or spacetime, which is thought to have
the advantage of collapsing certain distinctions to which the laws of
nature are insensitive, for example the distinction between open and
closed regions. (Sider 2013)
Arntzenius, F. (2008): Gunk, Topology and Measure, in: Oxford Studies in Metaphysics: Volume 4. Oxford University Press.
Sider, T. (2013): Against Parthood, in: Oxford Studies in Metaphysics: Volume 8. Oxford University Press.