I'm aware of Hume's argument and definitions of cause (though I'm very much open to any kind of discussion around the topic). But I'm most interested in how important his assertion that all ideas are derived from impressions is in his arguing about the status of causality, and establishing his definitions of cause. Would his arguments fail if his (potentially flawed) assertion was not taken as premise?
Hume characterizes causation as a "natural relation" (along with resemblance and contiguity) that we reason with to arrive at indirect and less than certain knowledge about the world. His argument for characterizing of it as a relation, as opposed to a "quality", does rely on his theory of ideas and impressions According to IEP, "Hume adheres to the Early Modern “way of ideas”, the belief that sensation is a mental event and therefore all objects of perception are mental. But causation itself must be a relation rather than a quality of an object, as there is no one property common to all causes or to all effects". The "Hume's fork", the surmise that all our knowledge derives either from matters of fact or relations of ideas, is also related to it. And let me mention in passing that Humean treatment of ideas as of a kind with impressions, but less "lively", is indeed highly problematic from modern perspective, see the Myth of the Given parts in How is the conflict between created by reason and external aspects of knowledge resolved?
But it does not seem to affect very much Hume's most famous skeptical argument about causation, to the effect that it is merely hypothetical. It relies more on his thesis that "reasoning concerning causation always invokes matters of fact. For Hume, the denial of a statement whose truth condition is grounded in causality is not inconceivable (and hence, not impossible; Hume holds that conceivability implies possibility)". Hume's fork is similar to Kant's later analytic/synthetic distinction, but the crucial difference is that, unlike Kant, Hume does not have "synthetic a priori" to fall upon, see Kant's “Answer to Hume”. So everything "synthetic" must be empirical, and there is no place for causation to come from other than from induction on experience. And this induction involves not observing any "causings" directly, but rather associating repetitive events, whose repetitiveness Hume ultimately attributes to what Mill later called the "uniformity of nature", see Norton's Material Theory of Induction.
This is how Hume arrives at his "definitions" of "cause" (more like conclusions about what it reduces to): it is a combination of contiguity, precedency and unity in which "the idea of the one determined the mind to form the idea of the other, and the impression of the one to form a more lively idea of the other". He hastens to add that "so imperfect are the ideas we form concerning it, that it is impossible to give any just definition of cause, except what is drawn from something extraneous and foreign to it". See When trying to identify causality, do we assume "nearness" between cause and effect? for comparison of Hume's and modern view.
It does not seem that ideas getting derived from impressions per se does much of a work here. Indeed, "Nothing is in the intellect that was not first in the senses" is a generic creed of empiricism, a.k.a. the "peripatetic axiom", which Aquinas ascribed to Aristotle, and which Kant, Peirce and Husserl also shared. One could arrive at a higher status for causation by rejecting it, and postulating some sort of Platonic mindsight or innate ideas (compare to Leibniz's amended "Nothing is in the intellect that was not first in the senses, except the intellect itself"). But there is no need for it. One can accept the peripatetic axiom but argue that our perception, although it requires the material of senses, is not fully sensual, so causation is either "framed into" it by reason (Kant), or is directly perceived/abstracted as a phenomenal aspect by some sort of "categorial intuition" (Aristotle, Peirce, Husserl). Conversely, one can allow any of the above and still argue that whatever else our ideas might derive from on other issues, in the case of causality they derive from induction on experience.
Reasoning about it simply, Hume's position that causation is - at root - a regularity in events is indeed based on his first-person epistemology wherein all ideas are built out of impressions (what we might now call "sensations"). The reason is just a matter of definition: if all events are built out of impressions, "A causes B" boils down to "event A is regularly followed by event B," then "A causes B" fundamentally just means "my experiencing the set of impressions A is regularly followed my experiencing the set of impressions B."
Consider a concrete example: you're playing Super Mario Brothers, something happens off-screen and you hear the sound of a 1-UP (a Humean impression) and see your status bar shows "4 lives" (another Humean impression), but you hadn't been paying attention to your number of lives so you don't know whether you just got an extra life (i.e., you just had 3 lives and now have 4) or if the sound just played but maybe you didn't get credit for the 1-UP because it happened off-screen.
Nowhere does the game or instruction manual say as a rule that that sound necessarily means your lives got incremented, and since the event happened off-screen you don't get shown "1-UP" visually; it's just that you have experienced that every time you heard that sound before (regularity) it has meant that your lives got incremented as shown in the status bar (Humean causality).
However, in another interpretation you may be asking whether you need the first-person epistemology in order to define causality in the Humean way. Like, would the definition of "A causes B" --> "event B regularly follows event A in time" still be tenable in, say, a physicalist view? I would say perhaps, but it depends upon your acceptance of the theory of relativity, because if you think that the timing of an event has no universally objective answer, you can't speak of universal ordering of events, then cannot speak of causality in the Humean sense. It in fact seems to me that it completely falls apart if you don't have a first-person perspective, as one person may say they experienced event B without experiencing event B, etc.