It's true that, under one understanding of "accessibility" and "determining", your accessibility relation is determined by your axioms. While that perspective is a valuable one, there is a more direct one in which the accessibility relation is just some subset of the cartesian product of worlds.
Neither of these perspectives are, I think, helpful to understanding the meaning of an accessibility relation. It is more helpful, I think, to talk about the sorts of applications to which they are relevant. One application is in the sort of modality that philosophers like David Lewis tend to employ, where we are able to quantify over all possible worlds. Here we are essentially in the simplest accessibility relation, "determined" by S5, wherein every world can access every other world.
On the other hand we might use a modal operator to express something about moral obligation, wherein ◻️P means one is morally obligated to make P true, and ♢P means one is morally permitted to make P true. Here, ◻️P does not imply P, and therefore the accessibility relation that models this modal operator will not allow every world to access itself.
So hopefully this helps clarify the answer: You choose the accessibility relation you want, to model the phenomena you want. It's like the choice between using natural numbers or real numbers--you can use either, depending on the sort of thing that you are trying to model. If you are counting things, you use natural numbers. If you are measuring or describing smooth processes, you use real numbers. If it is not entirely clear (maybe you have a tank of water mixed with a chemical, and it's not obvious whether you want to model every discrete particle of water using a natural-number system, or maybe you just want to treat these quantities like continuous variables in a differential equation) then there is a debate to be had.