Therefore, if the universe is infinite and it's possible to travel
through time and space near-instantaneously, by sheer chance there
must be a version of me out there that can do so and that wants to
contact me.
I think there's a central problem with this conclusion, which is that there are actually two conclusions you reach. One is that there are infinite doppelgängers with the power to contact their doppelgängers. The other is that because of the first conclusion, there are no doppelgängers which are not contacted by at least one doppelgänger. Neither conclusion actually follows from the premise.
The second conclusion actually contradicts your premise. If every possible thing that can happen does happen, then somewhere, there is a lonely doppelgänger who is never contacted by his fellows. If that's the case, we can safely assume that you're the you that no doppelganger wants to contact.
Or to put it another way, if I accept your premise, somewhere there is a doppelgänger of you putting a question on a version of StackExchange about whether there is a doppelgänger of itself that hasn't made contact with any of its doppelgängers.
Of course, there's no need to feel bad about being this lonely doppelgänger, since everything that can happen happens an infinite number of times, there are an infinity of lonely doppelgängers to fail to keep you company.
The first conclusion only follows from the premise if the probability that you exist is finite. However, it may not be finite, and instead be infinitesimal. If that's the case, there may be a finite number (even only one) of you, even in an infinite universe in which everything possible happens. I tend to think this is more likely than not, because there may be an uncountably infinite number of possible evolutionary paths.
The first conclusion is also vulnerable to the finite nature of time. It takes time for things to happen in. So it is possible that while it is possible to travel instantaneously, no species ever figures it out because they don't last long enough, in which case no doppelgänger of you ever has time travel.
The first conclusion is also vulnerable to uncertainty about what is possible. There is no civilization of spacefaring proto-lemurs anywhere in the universe because such a thing is simply impossible: to design and operate spaceships, you need a different mental toolkit than a lemur has. So the configurations of matter are not determined solely by random chance, it is random chance plus the laws of physics, which yields the laws of chemistry, biology, etc. The fact that lemurs can't build spaceships doesn't make spaceships impossible, it just makes them impossible for lemurs.
It is possible that a species that develops time travel has necessarily evolved biological structures which we lack, in which case there is no doppelgänger of you that has time travel, but time travel is still possible.
In conclusion, the only thing that you can conclude from the fact that you haven't been contacted by a time-travelling you-doppelgänger is that as far as you know you haven't.