I read the book "Caste", and can't recommend it highly enough. Are there any philosophical works that directly address the historical causes and current 'benefits' to society of what ended up resulting in privilege? Successfully? Any answers, beyond just trying to treat people more fairly?
Note: the question has been significantly altered since this answer was posted. Applied out of context to the current edition of the question, the below answer may be nonsensical or false.
Why do you assume that these ideas are rational, and not based off of ignorance, false beliefs, desire for social prestige, avoidance of short-term psychological distress, etc? Very few people, including intellectuals, on any side of any issue, are motivated by honest interpretation of the totality of available evidence.
What you want is introductory economics. Framing questions about behavior in terms of incentives and constraints is usually more useful than asking for defensible arguments.
NOTE: This answer was composed in relation to a previous version of the question, which has undergone a substantial revision (most of the original has been edited out). Please see edits prior to March 25, 2023 before downvoting.
This answer constitutes a shallow, layman response to an issue that attracts substantial academic interest, in fields including, but not limited to:
Recent discussion of privilege has centred around 'White Privilege'. For some examples of white privilege, see here. Even though it is perhaps not merely white privilege which you are most interested in, this link provides interesting examples of how privilege exerts itself in dominant cultures, in many places in the world.
But privilege of course permeates almost every aspect of our existence, across many levels of society, between almost any subset you care to identify. It exists in the relationships between ethnicities, socioeconomic classes, genders, sexualities, geographies, education and religion to name a few.
"But I of myself didn't create the circumstances I benefitted from.
No. You didn't. This is a key point. Most of us, regardless of our circumstances (particularly in developed cultures and communities) hold privileges that are bestowed upon us not on our own merits, but because of circumstance, circumstances which inevitably have historical roots; prior inequalities and injustices.
"Is that someone's fault?".
We cannot be reasonably blamed for the circumstances we inherit, but according to many, we can be blamed for not acknowledging the various privileges we inherit, and for failing to take steps to redress any injustice our privilege affords us; see San Francisco board open to reparations with $5M payouts.
"If I saw someone complaining about the weather in Alaska, I would suggest that they move to a better climate".
A reasonable suggestion, on the face of it, but it hints at the ignorance to which you frankly admit in your question; and ignorance of circumstances which prevent people from easily manipulating their circumstances, including by making a 'simple' move.
"Why then do some folks who live here already, think it is so bad? Instead of drilling holes in the ship, maybe people should try to improve it?".
For many, especially those who are growing up with an increasing awareness of privilege and phenomena such as intersectionality, I suspect they would clarify such a statement by acknowledging the relative excellence of many developed nations such as the U.S.A., but by simultaneously bemoaning the injustice, inequality and failure that pervade them; by regretting the potential of such nations is not more thoroughly accomplished. See OECD Better Life Index for relative standards of various nations by topic (interactive), and World Quality of Life ratings.
"Without the revolution of culture that began there [in the U.S.A.] about 500 years ago, it appears that life would be little better now than it was then. And for much of the world it is far worse. So, the folks who improved things are at fault? Why are we blaming the successful people?".
Much of the success of a relative few has indisputably be gained at the expense of many. It is possible to attack unjust privilege whilst simultaneously acknowledging positive outcomes that have been achieved by societies in which privilege exists. It is not an 'either/or' proposition. It is simply to say, "Let's acknowledge how certain privileges came to be. Let's celebrate our successes, but also recognise those who have not been fortunate enough to come along for the ride. Let's try to even the playing field a bit". One counter to this might be economic rationalisms such as trickle-down economics, in which inequality is 'justified' by claiming that "tax breaks and benefits for corporations and the wealthy will trickle down and eventually benefit everyone", in effect raising the 'poverty line'.
By calling privilege an 'invisible ocean', we may well be proving the point of those who are trying to illustrate just how privileged we are. Many of us are so accustomed to our privileges that we fail to see and/or acknowledge them, thus perpetuating/prolonging cycles of injustice.
I'll just throw a few philosophical works out there related to privilege.
Famously, economic privilege is covered by Karl Marx in Das Kapital. From WP:
Das Kapital proposes an explanation of the "laws of motion" of the capitalist economic system from its origins to its future by describing the dynamics of the accumulation of capital, the growth of wage labour, the transformation of the workplace, the concentration of capital, commercial competition, the banking system, the decline of the profit rate, land-rents, et cetera.
Thus, why someone might have the privilege of buying a jet and jet setting and why someone else may not have the privilege of eating is explained in terms of fiat currency, market forces, and fractional lending, among others.
Privilege conceived more broadly is discussed by John Rawls in his A Theory of Justice. From WP:
the author attempts to provide a moral theory alternative to utilitarianism and that addresses the problem of distributive justice (the socially just distribution of goods in a society). The theory uses an updated form of Kantian philosophy and a variant form of conventional social contract theory. Rawls's theory of justice is fully a political theory of justice as opposed to other forms of justice discussed in other disciplines and contexts.
Hence, privilege is discussed in the broader notion of social justice which covers ideas far beyond mere economics including ethical considerations.
Economy and Society by Max Weber is famous for giving a series of essays that explores notions, among others, of the monopoly on violence that the state has which might be seen as how the state reserves itself a privilege for administering justice. From WP:
Much like in other works, Weber makes it a point to mention the state's role in legitimizing and perpetuating violence. Weber writes that legal norms exist when they are kept through “...normally directly physical, means of coercion of the political community”. This legal norm can extend into further arms of the government and the public as described when Weber writes, “In the case of certain events occurring there is general agreement that certain organs of the community can be expected to go into official action, and the very expectation of such action is apt to induce conformity with the commands derived from the generally accepted interpretation of that legal norm….
Social contract arguments typically are that individuals have consented, either explicitly or tacitly, to surrender some of their freedoms and submit to the authority (of the ruler, or to the decision of a majority) in exchange for protection of their remaining rights or maintenance of the social order.
Berger and Luckmann attempt to describe the mechanism by which individuals are inculcated into a society including norms such as privilege in their The Social Construction of Reality. From WP:
The Social Construction of Reality: A Treatise in the Sociology of Knowledge (1966), by Peter L. Berger and Thomas Luckmann, proposes that social groups and individual persons who interact with each other, within in a system of social classes, over time create concepts (mental representations) of the actions of each other, and that people become habituated to those concepts, and thus assume reciprocal social roles.
John Searle explores how 'privilege' as a shared linguistic entity comes from processes in the philosophy of language in his The Construction of Social Reality (GB). What is a privilege can be explored by how society negotiates meanings to begin with.
Those are big philosophical works. I think you might find lots of exploration of 'privilege' in any body of work of critical theory (SEP) which explores why some have privileges over others which might be understood as domination. From SEP:
Because such theories aim to explain and transform all the circumstances that enslave human beings, many “critical theories” in the broader sense have been developed. They have emerged in connection with the many social movements that identify varied dimensions of the domination of human beings in modern societies. In both the broad and the narrow senses, however, a critical theory provides the descriptive and normative bases for social inquiry aimed at decreasing domination and increasing freedom in all their forms.