Capital punishment has seldom been seen as absolution. When the Church burned Witches their choice not to be absolved was seen as part of their punishment. But from a more objective position, as you point out, it is not necessarily punishment at all.
Still, it goes beyond that. People commit suicide, and we try to stop them. So this notion that emotional suffering is bad, and determines the value of life does not really hold water. The notion that being defined by an emotion is part of your penance is both too Catholic and not Catholic enough. It presumes those who commit crimes are more normal morally than we find to be the case, and it assumes our empathy applies or matters to them.
As Roy Royston has pointed out, it can be seen as vengeance. But it is more often simply a matter of avoidance. We want the bad man gone, and sometimes being held in a box is not gone enough.
Manson gets to torture his victims' families one last time by making the news with his marriage. They thought he was gone, but, no, there is more suffering to be had. He doesn't care, he is God, remember? He is just dying as fast as the law will let him so he can be omnipotent again.
It is surely not guilt that would define Tsarnaev, he considered what he did activism, and his willingness to suffer for it makes him a great man in his own mind. If he dwelt on this issue, he might suffer most from failure, not guilt. So if you let him out fifty years from now, he might be just that much more likely to do it again.
This is one of the reasons remorse is usually one of the criteria for judgement. The remorseful man will suffer more by living, and that suffering may make for actual absolution. The remorseless man simply remains a pointless risk, and can feed on his pride indefinitely to defend himself from his guilt.
If you know you have the right guy, and he is clearly not going to care, what is the purpose of just making him uncomfortable indefnitely?