From an initial analysis, your acquaintance is essentially putting forward a hypothesis about axiom selection; underlying this is the following premise:
What I know depends on what axioms I take.
I'm interpreting this premise as a claim that we as humans are doxastically axiomatic, by which I mean that it works something like knowledge based systems (KBS); that we have (for some indeterminate reason) a core set of axioms; and that we build our beliefs off of those by deriving them from those axioms, using some sort of logical system akin to how one would prove theorems in geometry from Euclid's axioms. Given this premise, your acquaintance is offering a particular reason we come up with the axioms in our analogous knowledge base... that reason being that it's simply an unjustified arbitrary choice. My objection to this idea begins with a rejection of the premise itself.
Regarding axiomatic beliefs, I would recommend watching (or reading) Richard Feynman's lecture The Relationship of Mathematics and Physics (here I'm linking to a certain point). The basic idea in the lecture is to emphasize the difference between an axiomatic approach, such as in Euclidean geometry, and what physicists do. Feynman argues, with clear examples, that physics follow what he's calling the Babylonian method... there are at least as of yet no fundamental laws (see @24:45). Per the analogy, the Babylonians simply had a bunch of facts they believed about mathematics; if they forget some they could re-derive them from others, but they didn't start with a core set... their beliefs were simply all interwoven. This is closer to how our own knowledge bases work. Furthermore, we have conflicting beliefs; even in science, there are known conflicts (Relativity and QM clash at certain scales); and though it's obviously possible to do so, it's not necessary for humans to commit to one set of beliefs as "core" or the other.
From a different angle, it's a bit naive to picture us as mere "theorem provers" in the first place, even in a Babylonian sense (appealing to Feynman's terminology here). We are observers, hypothesis generators, and theorem testers. We as humans don't simply sit around pontificating about the implications of our knowledge base... there's an actual point to all of this. We are agents. As agents, we are embedded within an environment, affixed with perceptive apparatus. A large part of our experience involves intentionality; we set about goals to attain desirable outcomes and act to attain these goals. Our strategy for goal attainment is based on models; roughly, world models (models of how things are and act) and "self" models (models of what we can do and what it affects). We incorporate this knowledge to develop action strategies to strive to attain goals. Both our world models and our self models are modulated by our perceptions; importantly, while we act, we monitor the actual result of our actions and can (and do) use that to measure whether or not, and/or the degree to which, we attain goals. One of our primary interests in this area is to maximize our ability to attain goals... to do this, we continually update our world and self models based on what works and what doesn't work (by comparing our predictions of what would happen if we do a certain thing to our perceptions of what actually does happen when we do that thing).
In other words, we attain knowledge based on playing with the world and based on what works and what doesn't work... that's drastically different from the proposition that our knowledge is simply a matter of pontificating the implications of arbitrarily chosen axioms. In a certain sense, we're deferring to the actual behaviors of the world we interact with our theories about the world; rather than simply imposing some theorems derived from some arbitrary axioms, we metaphorically "ask" the world what it's like by interacting with it, and derive beliefs about the world based on its "answers".
Furthermore, we as humans are to a uniquely intense degree linguistically social; we conceptualize and share knowledge with each other through language, this very conversation being an example of this. This introduces a large amount of conventional knowledge... by which I mean knowledge about things whose very existence is simply an aspect of belonging to the human project (e.g., who the town mayor is; how much money I have in my checking account, and so on). Again, here, there are a ton of aspects that we defer to the general conventions... if there's some doubt in my mind how much money I have in my banking account, I don't derive it based on a set of core axioms I choose to believe... I look it up or (at least once up a time) balance my checkbook. Even in balancing my own checkbook, I don't rely on my memory... I rely on my perception of handwritten notes... again, I'm deferring to signs written in a medium as a more reliable way of representing a historical account than recall (this in a sense is using language to communicate with our future selves about a "conventional fact").
In summary, your acquaintance seems to be applying a flawed analysis of how we work. The feedback loop between us and reality to apply/develop/attain world knowledge and self knowledge, and the feedback loop between us and other humans (and language instruments) to apply/develop/attain conventional knowledge, counter the notion that we work off of "unjustified" assumptions. Certainly many humans do have unjustified assumptions, but the primary core of our interactions defers to the actual entities the facts are about through perception or lingual processing to attain knowledge.