Dave's and James Kingsbery's answers are the best so far in the fewest words that gets at the crux of why a hard determinist would still punish criminals, but I wanted to flesh out some ideas a bit and explain why they make sense. I may go a bit overboard in explaining concepts but I want to make sure there is no confusion; having been a philosophy tutor and an assistant to a professor in a philosophy class, I know this topic in particular can be very difficult for people to understand, so a little extra meticulousness can go a long way.
A Slight Misunderstanding
From your question body, it seems you may misunderstand hard determinism slightly. Hard determinism does not give everyone "a license to do what they want", it can only explain why someone did what they did. It does not address the morality of acts — whether any act was "good" or "bad" (in layman's terms, it does not say that what they did was "ok"). Such a determination (whether an act is good or bad) comes from one's view of morality, which is — while perhaps later influenced by a belief in hard determinism — is not shaped by nature of determinism itself. That is, how we are raised, how we grow up, and what we end up believing in terms of "right" and "wrong" (our "morality")... while all those things are causally determined, the process of causal determination does not affect our views, it goes unnoticed by us. We in fact feel as if we have total free will, even at this very moment I feel as I have complete control of my body and will.
The only way hard determinism affects ones moral views is when someone gains an understanding of it (say, in school) and then applies it to their own life. But for example as a baby we are of course under the effects of determinism at all times, but such effects obviously have no influence on our beliefs one way or another. Only the outcomes, the byproducts of determinism affect us: that we ended up being raised by our parents or abandoned or taught to be kind to others or simply neglected — of course one's life circumstances (as occurred in this deterministic framework) greatly (wholly, in fact) determine how moral a person is in the end. The point is that you have to understand that the existence of a deterministic framework is separate from the notion of morality. In other words, if hard determinism is true, there are still (of course) people who are kind, people who are mean, good people, bad people, etc. Morality still exists in a deterministic universe. This is the first critical point to understand.
Hard Determinism and the Absence of Moral Responsibility
In a deterministic universe, the outcome of any events is causally determined by an unbroken chain of prior occurrences. We can explain anything, even human behavior, under this model. Why (for example) did little Johnny kill the cat? Well, he never had a cat growing up, his macho friends pressured him, he thought it would be cool, etc etc. He never made a "choice" per se, it was determined for him by a sum of influences. Can we then blame Johnny for his actions, if it was not in fact in his control whether to kill the cat or not? In a deterministic framework, no, we cannot morally blame him, as it was outside his control. However, in the end Johnny still was conduit for the final outcome determining the cats fate. We can blame it on his parents who raised him poorly, and further still on his parent's parents for not teaching them how to raise a child properly, and further back ad infinitum until we are blaming the big bang for starting it all. Clearly this gets us no where, and we are back where we started where we still think that even though it's not Johnny's fault, it was a bad thing that cat being killed (again, our moral judgement of an act is separate from the deterministic explanation why).
The simple truth of the matter is that although the dominoes did not fall in Johnny's favor (the ultimate cause was a chain of prior occurrences that led to him killing the cat outside of his ultimate control), he was the proximate cause (see an explanation of proximate and ultimate causes here). The reason why we have to punish the proximate cause is two-fold:
- If we didn't punish Johnny, he would never be taught that such actions are morally wrong. He may then simply continue killing cats, and we don't want that. Hard determinists still want serial killers behind bars because while we don't morally blame the killer, we would like people to stop getting murdered.
- If Johnny isn't punished, other people who see/hear about the event (the cat being killed) may then do it themselves if they see they can do it without punishment. In other words, it ends up being a contributing cause to future actions which lead to more undesirable outcomes (more killed cats). We don't want other people to think they can get away with killing cats, so punishment acts as a deterrent and helps prevent future cat killings.
This makes sense because people act under the presumption they have free will, that they control over their actions. Determinism then only changes the nature of the punishment in that a determinist should never feel vengeful; there should be no negative "emotion" (hatred, anger) in the punishment because as determinists we know it ultimately wasn't their fault. If any emotion is present, it should be sympathy; a feeling of pity and sorrow for someone else's misfortune that the dominoes happened to turn out this way for them.
A very interesting read on this very same topic is the story of Leopold and Loeb, and their criminal trial and defense by Clarence Darrow. See the wikipedia article, it is long but you might hone your understanding of this topic if you read the whole thing, and (I would say) it is very interesting in itself.
Update in response to OP's comments:
I never spoke to rehabilitation specifically, only empathy for criminals. So I agree with your statement "The heinousness of a crime shouldn't factor in" in principle, except insofar as we still have to punish people differently based on the severity of their crimes to send a stronger message that some crimes are worse than others. If you steal a candy bar from a store or download music (illegally), we obviously don't care about that as much as if you are a serial killer. We want to deter candy-bar thieves and music downloaders, but we want to deter serial killers MUCH more. While we could have one punishment for all crimes, we know from experience that people who commit what we would consider "minor crimes" can usually continue to lead ordinary productive lives as citizens if we punish them only a little and release them. If all criminals regardless of crime got life sentences, that would simply be an inefficient and wasteful system in which the goal is to keep the worst people (people who commit "major crimes") off the street and discourage certain behaviors in the rest of the population to keep them in line.
Don't get me wrong, I agree that a "one punishment for all crimes"-type system would make the most sense in theory, esp. if that punishment was rehabilitation rather than merely incarceration. However, another reason why a "one punishment for all" system would not work in practice is that our laws are imperfect and our enforcement of them is even less so. A lot of our laws were made by corporations and special interest groups and are in fact quite unfair to certain populations or just to everyone in general. The fact that gays cannot get married in many states is quite simply preposterous. Jailing someone for getting married to their gay partner in a "one punishment for all" system would be a terrible loss of justice, so different tiers of punishment at least ensure that there is less abuse there. Also, law enforcement in the USA is notorious for forging evidence and altering "facts" in order to convict people of crimes. There are dozens of well-known cases of this (for example, people getting out of prison after being charged with murder that DNA evidence later proves they did not commit) and probably hundreds of thousands of cases where evidence was tampered with by the police. A system where of only one punishment for all, presuming it was even remotely considered "just" for true killers and rapists, would grossly exacerbate this injustice.