Attacking infinite regress or complexity barely puts a dent in this argument.
But pointing out the fact that the divine could exist in an infinite regress, or that there is an infinite regress embodied in the divine, almost universally refutes Dawkins' fourth premise with no recourse that I can think of (except for some very specific descriptions of god).
Unlike the other popular answers here, I take no issue with the complexity vs. simplicity issue - and I would side with Dawkins on it. David Gudeman's answer mentions that this issue was "solved" by medieval philosophers, and while they certainly acted at the time as if it was solved, I would firmly reject that such a thing is in fact true.
The idea that god consists of no parts and is eminently simple, seems to involve a very non-instructive idea of what complexity entails in the physical world. What's "simple" about a thing (that also isn't a thing) that consists of no parts, exists nowhere while it also exists everywhere but consists of nothing, has nothing and is nothing (and simultaneously is everything), consumes no energy and yet can produce endless energy, and so on?
For all the reasons that manifest these kinds of impossible questions, a naïve, mechanical definition of complexity/simplicity must be wholly insufficient because it fails to describe, let's say, the difficulty with which the described thing can be understood fully.
As if a celestial block of tofu, in all its bland simplicity, would have power over seemingly unlimited creation across an infinite cosmos just because it only barely consists of any constituent parts.
I think Dawkins envisions complexity not as a measure of intricacy brought about by consisting of parts or internal interactions, but rather as the set of potentials, size of action space, number of paths of evolution, whichever term one may wish to use, available to the given system.
Let's say that any contingent physical system must have less total potential than its cause, otherwise you have a significant problem of explaining the evolution of entropy within both that system and its causal system (and possibly also conservation of energy, but to this particular discussion that'd be tangential at best).
If we imagine mapping the theoretical action space of a free electron versus an electron in a single atom versus an electron in a strongly coupled molecule, we can see that the more steps we go down the causal ladder, the space becomes lesser with every step taken. Under the idea we started out with, this means less complexity.
Further to this point, if we imagine that Gödel's incompleteness theorems express roundaboutly (and implicitly) that a proof generated inside any such described system has less possible paths of evolution than the system itself, then from here too it would be tautology that every proof made from any single system that adhere to the theorems is necessarily less complex than the system itself.
All this to say that the notion that God's (complexity or simplicity) is an open-and-shut, already-solved question, is one I would disagree strongly with.
However, that's not to say that the fourth premise is without problems
(4) The posited designer must be even more complex and hence even more improbable. But now we're back to 1. leading to an infinite regress
Strictly speaking, if we hold this premise true for the sake of argument, we haven't refuted the existence of one or more gods (though we could possibly have refuted a subset of specific descriptions of god) - we've just established that there isn't a first cause.
Dawkins' argument here, with or without the fourth premise, doesn't show that there couldn't exist an infinite regress of gods, each less complex than its "parent" - culminating in some not-the-most-complex god that designed or otherwise created the world.
I would agree that he does seem to imply this particular conclusion, but it's a conclusion that doesn't follow from any of the premises (not as it is described in this thread, anyway). Maybe a more full description of his argument carries some assumption about infinite regress, much like OP describes towards the end, but without that assumption stated as a premise the conclusion is still a non-sequitur.
Though I would venture so far as to say that even with the premise present, the argument will still fail because there exists no sound argument against infinite regress. The best you can hope for is the concession that infinite regresses don't have global explanatory power, but you can just as easily hold the same to be true for first cause arguments.
Stating that X exists by necessity and it's therefore nonsensical to ask "why" or "how" it exists, doesn't get you to anywhere that has actual explanatory power; it's an assertion of brute fact. So while you can invoke this argument and become able to terminate the chain of turtles, stated as the description of a logical argument that doesn't get you any further than validity - soundness yet escapes.
And as the astute reader probably has caught on to by now, you can get to the same state with the infinite regress; to validity, but not soundness. So the idea that infinite regress is "obviously" impossible or objectively not to be preferred, rests primarily (if not exclusively) on one or more fallacies.