Aristotle is said to have asserted that all fallacious arguments essentially commit Ignoratio elenchi (= ignorance of argumentation or missing the point for moderners). Since some logicians categorize ignoratio elenchi under the non sequitur (= does not follow), we could say that your observation that all fallacies are essentially non sequitur is right.
The problem with your assertion however is that scholars deeply disagree on the categorizations of fallacies and the scope of each fallacy. They cannot even offer a precise definition for 'fallacy': at most they agree that a fallacy is a defect in reasoning. Even Aristotle himself used ignoratio elenchi in the above broadest sense and a very narrow sense where the concussion simply is irrelevant to the premises, which is neither a straw man nor a red herring (e.g., "Abuse of the welfare system is rampant nowadays. The conclusion is obvious: we must abolish the system altogether.") Presently, Non sequitur is usually limited to the cases that produce a comical effect due to breaking the cliche or expectations: e.g., "My wife and I were happy for 20 years....Then we met."
Formal logic has been unhelpful to understand fallacies since many perfectly valid arguments are fallacious. Informal logic that is informed by critical thinking, rhetorics, communications and AI have been far more helpful to understand the categorization of fallacies. Under the informal logic,
it is now clear that many fallacies are fallacies for the reason other than relevance (broadly, non sequitur): e.g., weak induction, illicit presumptions. Viewed in this light, your assertion can be said to be false.