The principle of explosion is a feature of classical logic. It is baked in. You cannot remove explosion from classical logic without turning it into a completely different logic.
There are many ways to prove explosion and you would somehow have to block all of them. It can be proved from disjunction introduction and disjunctive syllogism as you show. It can also be proved using reductio ad contradictione, using conjunction introduction and elimination, from the rule of implication, and in other ways. It can also be proved using model theory and using modal logic.
There are non-classical logics that lack explosion. These are collectively known as paraconsistent logics. These include the entire family of relevance logics, minimal logic, dual intuitionistic logic and the logic of paradox. Some of these are motivated by the desire to tolerate the possibility of true contradictions. Others are motivated by a requirement for the premises of a valid argument to be relevant to the conclusion in a specific way. So if you wish to avoid explosion, you can, by using a different logic.
But why would you want to avoid it? In classical logic no contradictions are true. In fact, in modern usage, the term 'contradiction' has come to have the conventional meaning of a logical falsehood. So within classical logic, explosion serves only to show hypothetically what would follow if we were to suppose something that is inconsistent. A contradiction would have the bad consequence of making a theory trivial by proving every proposition and its negation. Far from being a defect in logic, explosion acts as an enforcer. It insists that you respect the law of non-contradiction on pain of triviality.
As someone who likes to use non-classical logics, I can tell you that logics without explosion are more difficult to work with than classical logic. Also explosion is useful in mathematics. If a theory is inconsistent, then by explosion all sentences within its language are theorems. So by contraposition, if there is even one sentence that is not a theorem then the theory is consistent. Gentzen used this to prove the consistency of Peano arithmetic by showing that with some modest assumptions, it is impossible to prove 0=1.