According to my knowledge, a main criticism to dualistic theories is that they give the mind a special characteristic in contrast to matter.
But is this true, since matter is also split up in several basic (physical) families of different components, like e.g. fermions, bosons, space-time, ...
So the mind and stuff related to it could also just be another family.
Who else worked on this?
It is not exactly like that. Dualism talks about two different fundamental substances i.e. mind and matter. There is a gap between these two substances. How do they react one on the other? Monism in the varieties of idealism or materialism solve this problem by assuming one fundamental substance. Idealism presupposes everything is an idea, and materialism everything is matter. It's a discussion that goes back to the ancient philosophy. Plato's treatment of the case give rise to the statement "only the same can acknowledge the same".
The different "families" you mention are different qualities of the basic substance.
For dualists, mind substance is supposed to be fundamentally different from matter in a number of ways. Two of them:
Mental states can be known directly and with certainty: A person just knows that they are in pain, in love, or that they believe in something specific. Material states don't share this property: we can never know them directly the way we know mental states, we can only infer them indirectly from our senses or from communication with other people.
The identity argument: If mind and matter were the same, then anything that is true about one is true of the other. Since there are things which are true about the mind, but not true about matter, then they cannot be identical.
The key point for your question is that the distinctions between mind and matter apply to any kind of matter (fermions, bosons, quarks, waves, particles, force vectors,...). So mind substance can't be just another family of matter, it has to be something altogether different.
Saul Kripke, David Chalmers, Thomas Nagel are all modern philosophers who argue for this fundamental difference. Frank Jackson was for while did as well, with his famous "Mary the color blind scientist example", but has since changed his mind.
There are many philosophers who argue against dualism, again I will state two. They don't argue so much for mind being another type of matter, instead they argue that dualists are wrong about their being fundamental differences between mind and matter in the first place.
Daniel Dennett argues tries to show that when examined closely, mental states are not as different from physical states as we think they are. For example, he challenges that sensations and beliefs are as certain as we think they are. That we can be just as wrong about our beliefs and sensations as we are about the material world around us. Search for Daniel Dennet "Experienced beer drinker" experiment and his ideas about inverted goggles.
Paul Churchland goes further and claims that mental states don't exist at all. Everything in the world is physical, and the only reason we need mental descriptions is because our science is not advanced enough to describe our brain adequately, in the same way that ancient people used magic and powerful gods to describe physical and astronomical phenomena that now we can explain using purely physical facts.
The philosopher David Chalmers proposes this sort of thing in his Book The Conscious Mind, p.126:
Although it is a variety of dualism, there is nothing anti-scientific or supernatural about this view. The best way to think about it is as follows. Physics postulates a number of fundamental features of the world: space-time, mass-energy, charge, spin, and so on. It also posits a number of fundamental laws in virtue of which these fundamental features are related. Fundamental features cannot be explained in terms of more basic features, and fundamental laws cannot be explained in terms of more basic laws; they must simply be taken as primitive.
...
Perhaps we might take experience itself as a fundamental feature of the world, alongside space-time, spin, charge, and the like. That is, certain phenomenal properties will have to be taken as basic properties. Alternatively, perhaps there is some other class of novel fundamental properties from which phenomenal properties are derived. Previous arguments have shown that these cannot be physical properties, but perhaps they are nonphysical properties of a new variety, on which phenomenal properties are logically supervenient. Such properties would be related to experience in the same way that basic physical properties are related to nonbasic properties such as temperature. We could call these properties protophenomenal properties, as they are not themselves phenomenal but together they can yield the phenomenal.
As a note, his conception of protophenomenal properties reminds me of Leibniz's Monads, and indeed Chalmers kind of splits his time between being a Property Dualist and a Panpsychist.
