Looking into the epistemology of modality would be a good starting point. More specifically in relation to mathematics, Hamkins and Linnebo [17] go over a "potentialist" account of mathematical (or at least set-theoretic) ontology, which roughly hearkens back some to Zermelo's own "unfinished totality = potential infinity of actual infinities" viewpoint. You could also try out Zalta's modal neo-logicism:
Zalta (1999) proposes an interestingly different, because modal-logical, route to the natural numbers. ... The first-order Barcan formula [as used by Zalta] forces one to interpret quantifiers as ranging over all possible individuals, whatever world one is ‘in’—no ‘expansion’ or ‘contraction’ of the domain can be involved as one traverses the accessibility relation from possible world to possible world.
The logic is free, and descriptive terms (the description operator ι is primitive) are interpreted rigidly—that is, the denotation of a descriptive term in the actual world, if it has one there, is its denotation in any other possible world.
Now, so far as (deductive) logic is often characterized in terms of the necessity of conclusions from their premises and by the inference rules, there is a "modalization" of logic-talk from the get-go. Induction then might be handled by interpolating possibility-talk and probability-talk; in Łukasiewicz logics, for example:
This revolutionary development came in the context of discussing modality, in particular possibility. ... For if values other than “0” and “1” are interpreted as “the possible”, only two cases can reasonably be distinguished: either one assumes that there are no variations in degrees of the possible and consequently arrives at the three-valued system; or one assumes the opposite, in which case it would be most natural to suppose, as in the theory of probabilities, that there are infinitely many degrees of possibility, which leads to the infinite-valued propositional calculus.
Will circularity really be avoidable, though? Consider impossible-worlds talk:
... another definition has it that impossible worlds are worlds where the laws of logic fail (where these may be theorems of the target logic, or its logical truths, or valid consequences, etc.). This depends on what we take the laws of logic to be. Given some logic L, an impossible world with respect to the L-laws is one in which some of those laws fail to hold (see e.g. Priest 2001, Chapter 9).
I've even seen people express doubts over whether modal logic itself is "really" logic. But so now rather than think of all domains of discourse as layered in a hierarchy, one can imagine them as sections of a sphere, and we might want to identify both the union of all the domains, as the entire sphere, and then also the intersection of the domains. Does logic occur as the intersection, here? Or does language? If both logic and language in different ways occupy the intersection, perhaps we could hold to both "logic is about objective possibilities" and "logic is about subjective linguistics" (c.f. Kant's critique of metaphysics, which is much invested in judging modalized knowledge claims).