I don't think this is enough to rule out a block universe where the past, (and future if you desire), are equally real to the present.
We don't know that time is really continuous or not, but I don't see how a infinite (not necessarily continuous) worldline of a finite number number of particles is problematic. There some things we know are discrete and some we think are continuous. Having both isn't a priori a problem.
What would going back in time actually mean? If we take the forward arrow of time to be the second law of thermodynamics, then does an entropy descreasing action mean we've reversed the arrow? Not really because there must be an equal or greater entropy increase somewhere else in the universe and the macroscopic entropic arrow still points forward/toward increasing.
But with perfectly symmetrical physical laws (which we do have, nothing has violated the combined CPT
symmetry), why can't I reverse my local entropy to the point I am young again? Well in theory you could. Every trajectory has a CPT inverse, and as long as the overall entropy of the universe keeps increasing, any local decrease is totally fine and happens all the time. The engineering to de-age a human is completely out of reach but not inherently impossible.
But this isn't going back in time I would say. It is reversing the local entropic arrow, but not changing the overall second law applied to the whole universe or undoing anything. The past would have still happened, you just locally engineered a mini-recurrance basically, but you never traveled "back in time".
The overall entropy still increased and the second law is happy. Nothing violated the second law and you 'traveled in the reverse entropic direction' to a configuration you had in the past. This may make time travel less interesting, but it isn't time travel really. The more interesting point to me is that we have totally symmetrical laws. Why there is macroscopic direction (toward overall increasing entropy) is only explainable with an additonal claim, that we started in a incredibly low entropy state, the big bang. At least that is the normal framing. There are other theories that don't require the universe to have been in such a state for the 2nd law to emerge from symmetric laws.
So what did the universe's and your worldines look like during this? The universe never looped back into a previous configuration. We think it is unbounded, so there are no reccurances. You never traveled 'back' time, but you did travel on a 'backwards' entropic arrow while de-aging if that makes sense. Did you loop back to find your old body in space? No. But that's not because the block universe doesn't exist. All of the above happens in either case. The block universe comes from our perception of the flow of time. Maybe every "instant" is just a slice like a frame from a movie, and the universe is just some higher dimensionsal stacking of these "instants" and the feeling of time passing is a conscious illusion. The stack of timeslices is static and unchanging and always existed, and does have an ordering. You didn't time travel to a previous timeslice, you just progressed to a new timeslice yet you lowered your local entropy (the de-aging bit).