Was it just a coincidence because he simply never got the opportunity to, or because he refused to leave his home city for ideological reasons ?
This doesn't really answer your question in the terms given; but it might throw some light on it.
Arendt wrote in her short book, The political theory of Kant:
How serious Kant was about the enlargenment of his own mentality is indicated by the fact that he introduced and taught a course on physical geography at the university.
then she goes on to say:
he was an eager reader of all kinds of travel reports; and he - who never left konigsberg - knew his way around both London and Italy; he said he had no time to travel precisely because he wanted to know so much about every single country.
I believe he did leave Koningsberg twice, both times for tutoring in the period after he had graduate from university but before he got a position in it. But I don't believe he ever left apart from for those two occasions for the reason of tutoring.
King Louis XVI of France never left his Palace at Versailles during his life (except to be guillotined in Paris in 1793). People just did not put much value in travel at those times. The idea that brainlessly moving around has any usefulness is fairly recent.