Maybe I have to write an answer myself. But this is not really an answer, it is more the background which caused me to ask this question. But it answers some of the objections brought up in comments, even if it could and should be probably elaborated much more thoroughly. (Sorry for misusing the "community answer" feature in that way.)
After reading simultaneously in a number of German introductory quantum mechanics books, I suddenly noticed that not a single one had the courage to say that a Hermitian operator is in general only partially defined on the corresponding Hilbert space. The theory of rigged Hilbert spaces seems simple enough to me, so I started to wonder where this overwhelming fear against partially defined operations comes from. It's a bit ironic too, considering that quantum mechanic itself is so fond of some interpretation which make claims about some sort of fundamental undefinedness. The fear seems to come from mathematics itself, i.e. mathematics is unbelievably dismissive towards partially (un)defined operations. Then I thought a bit about whether the theory of rigged Hilbert spaces could me made even simpler, if the corresponding mathematics embraced partial undefinedness even more fully, and that's the context where families of semi-norms, partially defined norms and partially defined semilattices seemed to paint a simple and beautiful picture.
I had struggled before how to best get the point across that semilattices are different from lattices, and that turning every semilattice into a lattice by "adjoining" the "missing" elements doesn't make live easier. I'm still working on a theory that has a generalized implication operation in addition to meet and join, that allows to "rescue" the original semilattice structure, even if it has been compromised by adding "unnecessary" elements.
If we have an axiom of choice like in ZFC, then every partially ordered structure can be embedded into a complete Boolean lattice. Hence it is pretty clear why we can get away with the LEM, if we really want to. But then we also have to accept all the consequences, i.e. infinite structures with unbelievably big cardinality. But as already the fact that a logic can be seen as a preordered structure where the derivability is the order relation between different propositions or formulas seems to be unbelievably hard to swallow, this logic based approach to these question is likely to be unconvincing.
Both infinity and undefined operations have much more mundane applications where they make life significantly easier. If we adjoin the points at infinity to an affine space, we get a projective space with much nicer properties than the corresponding affine space. But if we adjoin infinity to the possible values of a norm, we don't really do ourself a favor. If we instead allow a norm to be only partially defined, we seem to really get something nice in return, similar to the projective spaces, but of course different. The poorest current "loser" from the fear against partially (un)defined operations are probably inverse semigroups, and semilattices are a special case of inverse semigroups (i.e. idempotent and commutative).
One nice example here would be how an inverse semigroup can represent an equivalence relation with essentially the same amount of information as the corresponding binary relation, while a group requires exponentially more (artificial) elements, if it doesn't want to add structure to the underlying set that wasn't there before. This would also partly explain what I mean by trading a finite context for an infinite one. If I require exponentially more (artificial) elements than the natural numbers, than I have to cope with the continuum. The natural numbers are still a finite context in a certain sense, but the continuum is definitively an infinite context.
The connection between the law of excluded middle and partially defined operations is probably hard to appreciate without some background. Sadly, this background might be slightly too mathematical for a philosophy site, but let me add it nevertheless. A Boolean algebra is a distributive lattice with an involutive negate operation. A lattice is a partially ordered set where any two elements have a greatest lower bound (meet) a least upper bound (join). A semilattice has only one of these two operations guaranteed to be defined everywhere. A semilattice can also be defined as an idempotent commutative inverse semigroup. And an inverse semigroup can be represented as a subsemigroup of the partial one-one transformations of a set.
The connection to the law of excluded middle is that often the structure of a complete semilattice (among propositions, formulas or sentences) arises naturally, but a complete semilattice is nearly indistinguishable from a complete lattice. But you can define an implication operation ("from A follows B" for a Heyting algebra, but in general rather "from A, B, C, ... follows Z") which allows to distinguish "and" and "or" (or "meet" and "join") properly. This is important, because the symmetry between "and" and "or" so typical for classical logic is often just an illusion caused by the fact that complete semilattices are so hard to distinguish from complete lattices. But if the maximal element (infinity) is removed from the complete semilattice, then it becomes much easier to distinguish it from a complete lattice.