Caveat
I have formulated a Short Response and posted it as a second answer for those who see this response as TLDR.
Short Answer
This argument is not strong because it betrays typical usage for 'brute fact'.
A brute fact is one which admits no explanation for why it's true, even in principle.
While a brute fact is a fact without explanation, it is very possible to adduce, say, a scientific explanation to make a brute fact a scientific fact. A claim can be both a brute fact and a scientific fact at the same time. Language is highly contextual.
...we conclude that there are no brute physical facts.
You've reasoned your way past common sense. It's simply a brute fact that there are brute facts. Let's look closer.
Long Answer
There are two primary flaws as I see it in your informal argument, one among the premises, and the conclusion itself, so let's see if we can explain why the argument you provide isn't strong or cogent.
First, if we consider the premise that a brute fact can't have an explanation, we can undermine an important foundational element in your rhetoric. Let's simply admit that a brute fact and a scientific fact aren't mutually exclusive. That's easy to do knowing that the function of the sciences as an empirically epistemological tool is to take brute facts and transform them into justified, true beliefs of a sort. Thus, that 'Snow is white' is a rather atomic, brute fact known to 2 year-olds everywhere, a brute fact can be bolstered in an epistemological sense by citing scientific theory to explain it. That explanation the science of optics, which includes facts and theories about light and perception, now provides. Thus a brute fact can also be a scientific fact. What started out as an intuitional claim thus has been vetted by the global and historical community of science. Thus, we have a brute fact with an explanation.
Brute facts generally have explanations, even if we're not aware of them when we make a claim that is a brute fact. What makes a claim a brute fact is whether it comes from a thinker's intuition or not. Thus, "Snow is white" is both a brute fact and a scientific fact depending not on the utterance or proposition, but it's place in the worldview of the claimant.
Second, you seem to think past what a brute fact is. A brute fact is simply a self-evident truth. 'Gravity pulls things down.' Brute fact. 'Things in motion come to a stop.' Brute fact. 'The sky is usually blue.' Brute fact. The traditional use of the term brute fact means that on the threshold between what we perceive and what we can claim, certain claims seem to be made universally by people everywhere, and therefore brute facts are a universal human experience and undeniably exist. In a naive sense, a brute fact is simply a nearly universally consensual agreement about the state of affairs. It might help to think of brute facts as natural foundational claims in the sense of the Agrippan Trilemma. What is a brute fact is subject to fuzzy logic and categorization, of course, and near-universal is necessary to qualify since a color-blind person wouldn't consider claims about colors brute facts.
But brute facts are a brute fact because human beings when they reason are capable of making simple, obvious claims about the physical world of which they are apart. That naturalism has led to various scientific methods that substantiates the brute facts as scientific facts based on empirical evidence isn't a forgone conclusion, and many cultures today simply don't have anything nearly as sophisticated as the sciences, and rely heavily on brute facts. The Yahgan people are good example of a culture where this is true. This is because brute facts are a product of our naive capacities, such as naive physics and folk psychology which are a resultant of our philosophical intuitions. In fact, sometimes brute facts turn out to be wrong.
Consider the claim the brute fact that 'Things in motion come to a stop.' This is a famous brute fact that was elevated through philosophical argument as a certain fact in Aristotelian physics. Not until Newton was the 'truth' of this claim undone. Now, we have a scientific fact which contradicts the brute fact. 'The existence of an object's inertia is nothing more than a claim a thing in motion tends to stay in motion until stopped by a countervailing force.' Does that mean that 'Things in motion come to a stop.' isn't a brute fact anymore? Of course not. Any child when asked after this principle is likely to make the former and not the latter claim. Why? Because human reason is defeasible and built on experience which begins as highly intuitional and never fully escapes intuition. Utterances and symbols derive their meaning from intuitions, ultimately.
So, not to mince words, your argument appears to have all of the hallmarks of a valid deductive argument, but in reality this is a masquerade of sorts since it is a weak argument based on premises that both contradict common-sense observations about what a brute fact is defined and which arrives as a self-evidently contradictory conclusion regarding the universal prevalence of brute facts as an aspect of human experience.
ADDENDUM 2022-02-06 - The Importance of the Duhem-Quine Thesis
I've received the criticism and respond.
First, the definition of brute fact can indeed be constrained to a narrow interpretation depending on how one views epistemic modality. I do not believe that Searle, Quine, Duhem, or others use it in the way you do. Let's clarify the narrow view you seem to advocating:
A brute fact is one which admits no explanation (no matter how contrary to physical experience and reason) for why it's true, even in principle (in no possible world perhaps even considering non-modal realism).
Whereas my experience seems more inline with what WP's article claims about Duhem's view:
A brute fact is one which admits no explanation for why it's true, even in principle (although the provision of one or more scientifically feasible explanans might dissolve the brute fact, though there is no non-defeasible measure of which explanans is best given the theory-ladeness of scientific theory).
Thus, we have two disparate worldviews at play, with the instrumentalism-realism debate rearing its head again. I simply assert that your views on naturalism are inconsistent with contemporary views on the nature of the defeasibility of reason and Duhem-Quine thesis.
Let's adduce a further passage from the article:
To reject the existence of brute facts is to think that everything can be explained.
This, of course, is your objective in your argument, to reject the existence of brute facts, and I think we're moving towards the same ends, but your argument again has an obvious deficiency. Must because linguistic facility allows us to explain anything doesn't mean that an explanation is adequate, and there is the rub where we split hairs. You seem to think that ANY explanation, no matter how absurd, inconsistent, or constructed willy-nilly disqualifies a fact from being brute, and on the face such a linguistic category 'explanation' certainly includes these members, but there's a wholesale effort to be blind to the obvious truth that there are grades of membership involved in explanation!
'It is raining.' is a brute fact, but should I accept the explanans that it's because 'Angels are crying.' satisfies the criterion of ADEQUATE explanation? Of course not. It's an absurdity. A fiction. A story told to fill in the gaps of knowledge of children in the Judeo-christian tradition usually rationalized with admonishments about the importance of faith. No. This simply won't do to disqualify a brute fact. An explanation must adhere to the best practices of the sciences (read epistemic methodologies used to reduce uncertainty) and we can use simple philosophical razors to dismiss bad and complicated explanations rendering them effectively non-explanatory.
Let's provide a more insightful example. Through complicated appeals to empirical evidence and an extensive exercise in computation, we might arrive at a brute fact. 'Subatomic particles are the ultimate constituents of matter.' This brute fact goes back to the ancient Greeks, and even under the current standard model, seems to be a fundamental truth, one of two which is contradictory under QM. Now, the question remains. Is there or is there not an ADEQUATE explanation for this fact, and I would posit simply no. It's a reasonable assumption about space-time and has been for thousands of years, but no ADEQUATE explanation exists, at least for me. I'm a conventional naturalist who rejects alternative physical realities such as the multiverse, Heaven, Brain in the Vat, the Simulated Universe and the Realm of Forms. (They're all lack any empirical validation.) For me, the universe simply is divisible into subatomic particles and it is a brute fact.
So again, your definition only works if you exclude the relative nature of what constitutes a fact. I do not believe post-Quine, that this is how the contemporary usage of brute fact applies, at least through my superficial readings with Searle, and I do not believe that the class of explanadum should consider any linguistic construct which is grammatical and barely sensible. This is not how science is done, nor how contemporary logic works except in the toy problems of logicians who generally insist that the traditional laws of thought must apply. It violates intuitionism, constructionism, and ignores the import of the discriminatory capacity of the thinker to decide what qualifies as an acceptable explanation.
Now, that being said, let's drill down one last time. You say:
Since all brute physical facts are contingent physical facts, they admit a further explanation as to why they're true, even if just in principle.
No empiricist in his right mind would accept this claim, because while all brute physical facts are contingent since they are constructed by thinkers, a theoretical criteria for the existence of an explanation does not satisfy the primary criterion of empiricists since Hume, that is the satisfaction of truth by empirical evidence. What. Because a scientist can think that the the universe is a simulation, the fundamental nature of subatomic particles as bits in a metaphysical computer means 'subatomic particles as fundamental' is no longer a brute fact? I would say to such a claim, dream on. This is exactly the angels-on-a-pin-head metaphysical debate that, while an interesting exercise in logic, has no bearing on the more certain world provided by the sciences. So, because a butterfly can dream us into existence there aren't inexplicable facts?!? Come on. You don't actually believe that to be true, do you?
No, an empiricist and by virtue scientist has to put her boots on the ground and draw a line between physical and fanciful, and that's exactly what science purports to do. To reject the existence of brute facts is to reject the efficacy of science, and to lack any criteria for differentiating between the natural and the supernatural, and I suspect few members of the NAS would subscribe to such a philosophy.