Physically speaking words can obviously be hurtful. Just utter them loud enough or in a weird frequency and they can do real physical damage. Sound is not immaterial, but a real existing pressure wave. Also you could construct a language where a word happens to trigger an earworm or whatnot. And following that thought we could transfer that to the question of whether information can be hurtful and if you look at, for example computers, then yes information i.e. worms, viruses, spam, etc, can be physically harmful without requiring physical access to violently mistreat a device.
Though to be fair that's usually not what people mean when they talk about that. so you're mostly right, that in most cases it's not actually about the word itself, but about the act of speaking it, the context in which it is said, how it is likely received, the message that is being plain or encoded and so on.
Messages can only hurtful when they reach those they are intended to reach - the 'abused ones'
Not really. It's fairly easy to set the context for an abused person without actually addressing them directly. Idk self-fulfilling prophecies of gossip, like you know how "@gaazkam is always so aggressive and rude?" Like if enough people get such a framing they are more likely to perceive your actions through that lens, that is as aggressive and rude and act accordingly even if they were never intended as such, to which you are more likely to act accordingly (aggressive and rude) and hence you've fulfilled the prophecy, that you were never aware existed in the first place. So they technically said the truth when they gossiped, right? (Well no, but good luck proving that). And depending on what the framer is saying about you that can be very harmful, while your agency in this might not even be active, like you might not even know what is played.
Like if another person frames you as "the enemy" then it doesn't matter whether you are polite, rude, outgoing or shy or whatnot, it's either a description of your bad character or a ruse to fool them, which is again even more indicative of your bad character.
But enough about being indirectly effected by speech. What about direct verbal assaults? Well you seem to focus heavily about the inherent aggression in the language like if someone uses an insult or whatnot. But it's again usually about the explicit, implicit or pretended context. So idk an insult from a stranger with no consequences can often easily be ignored and insult from a friend hits much harder or not at all depending on whether it's genuine or in jest. So context matters a lot.
Also if it's not 1 stranger (what a weirdo), but 100 strangers, then you're likely to experience that as a hostile environment, while each of the strangers might see themselves as individuals releasing steam and would think that they are the exception not the rule. So the problem with "amateur jokesters" is that they rarely are aware of the context of other people or take the time to set up the appropriate context. Also sometimes what you think is harmless, just isn't. Just because workplace harassment used to be normal doesn't mean it ever was ok.
And "growing a thicker skin" is asking for complacency with a hostile environment, which is basically making the argument of "this hostility isn't going anywhere so it's YOU who has to change yourself to fit". And rarely is that talking about a natural environment that actually can't change and more likely it's about an actually hostile environment that should have changed long ago. Like being able and even encouraged to share their perspective of a situation should not be seen as a threat, but is necessary to avoid miscommunication, but apparently many still believe that's a one-way thing. The longer something "works" the less likely people are willing to accept that it never actually worked for everybody.
So no reaping outrage and flak for a mindless statement is not always a bad thing but can be constructive criticism that you've crossed boundaries that you were not supposed to cross. Now whether that has to end in total personal destruction is a different topic, but that it happens in at all rather than sucking it up, isn't necessarily a bad thing.
Though that being said it's vitally important to have rules mandating equal rights and to have them be reliable and to be able to report these things. Because usually a hierarchical structure to a person at the bottom looks like a monolith, so if the direct superior only responds with "that's your problem, toughen up, grow a thicker skin", then you are likely to perceive this whole environment as toxic and yourself as on your own, meaning you're either shrinking as a person or your overly aggressive to compensate the lack of social power in that environment. So being aware that you don't have to suck it up, that you have rights and that you can count on that is very important.
Again whether personal destruction and 1 strike policies are a good implementation of that or whether companies are just very cautious of shitstorms to the point of breeding their own hostile work environment, is a different question.