The problem is that pretty much every action in the trolley problem is unethical.
You are about to make a decision which will cause the death of at least 1 person and you'll be aware of that consequence and do it anyway. So technically you are the one sealing their fate and killing them. it's not an accident, but homicide, despite being caused by an accident. Likewise it's most likely not murder and you have a good chance to defend yourself against accusations of guilt and deliberate wrongdoing.
But if your moral compass includes "Don't cause other people's death if it is preventable" then you're screwed. Because either death is preventable by pulling or not pulling the lever. However that causes another preventable death and so you'd be toggling the lever as fast as you can until the trolley passes the switch. Which is essentially equivalent to flipping a coin.
Though technically both that and flipping the coin are still cop-outs because at the end of the day it's not the coin or the end position of the lever after toggling that determines the outcome it is you deciding to accept that outcome and not overturn it. You didn't give up that agency you simply chose to ignore it, which is still a decision.
Like if you told anybody, with the clam rational expression of someone thinking they did everything right, that you flipped a coin, they still look at you with an expression of horror and the explicit or implicit question of: "You did WHAT?"
And it will probably immediately dawn on them that it was still not the coin but you who made the decision and that you just try to pass on the responsibility to the coin, which likely won't work.
So as said there is no ethical action in the trolley problem. Either way you did something wrong, but it's unsatisfying and problematic to say you did something wrong without being able to say what you should have done instead and if you can't do something right, can it be even called "wrong".
So you could tinker with variables in that scenario. Like you could argue that you head no agency. That a choice between two bad options is not a choice.
In that case you're best option is to be less wrong. Which would prompt you to quantify and compare the options. But according to what metric?
As you can't infer anything about the specific people you just have their numbers. Now what? Are humans differently abled and the bigger group has a bigger chance for extraordinary talent? So going by some sort of stochastic utilitarianism, which could very well be wrong even according to it's own metric. Are all humans equal? But then you'd also have to answer equal to WHAT. Like do you assign a value of 1 and argue
1*5=5 > 1*1. But you could also that the value of human life is unmeasureable and infinite so
1*infinity = 5*infinity and you're back at your coin flip. Or probably uncommon but you could also argue that the value is negative because of negative human impact on the environment and so
-1*5=5 < -1*1.
Or you could question whether that don't do preventable deaths is to be dropped, but most people like that for a reason. Or you could focus on "being the cause" and not act and thereby ignore your agency in being the cause, though as said that doesn't really work either. Or you could argue with harm reduction, though that again is subjective and reliant on information that you don't have. Or you could hold a referendum ask people for their opinion and go with the majority. Which is why I think asking about the trolley problem itself is unethical.
There are a ton of different paradigms that you can test and look how they would play out with the trolley problem, which is what makes it interesting. So you can make a decision (literally any one of them and defend it), but you can't really find a solution (inside the box), because none of them would strike me as good or ethical.
The only real solution that would also be a cop-out is if you died of a stroke or fell into paralysis upon the weight of that decision without being able to act at all. That is to naturally lose your agency. Then it's truly not your fault and genuinely an accident. But if you could prevent that by keeping your calm you'd likely ethically be required to do so, setting you back to square one.