Category Theory is indebted to [structuralism] in mathematics, particularly the idea of a homomophism which is a structure preserving map which first appeared in algebra. There was a parallel movement of structuralism in the humanities which can be traced back to the structuralist linguistics of Saussure and semiotics.
It was inaugurated formally in the 1945 paper by Maclane & Eilenberg General Theory of Natural Equivalences, which was amusingly enough dismissed for not having any content. Maclane also borrowed the terminology of Category from Kant.
The focus is on relationships between things and not the thing-in-itself (this is not Kants noumena!). This still allows probing the thing or object. For example to find out what elements an object has one examines the totality of all relationships between a point object and the object itself. An alternative way to think about this, is that one can discern two ideas in set theory - that of sets and of functions. Here sets are privileged over functions since functions can be expressed in terms of sets. One directly sees that a set has elements through the membership relation. One could say that the ontology of set theory is sets. Contrariwise Category Theory places both functions and sets on an equal footing and dismisses the membership relation, one discerns the structure of a set through functions, in this way the membership relationship can be recovered. In fact its also possible to do with the idea of sets altogether and retain only the idea of function.
Hence, one can term Set Theory as analytic and Category Theory as synthetic (or holistic as Drossos uses in his paperSets, Categories & Structuralism).
In (naive) set theory, the unrestricted use of (the axiom of) comprehension, that is asserting the existence of a set satisfying an arbitrary predicate leads to Russells paradox, (although this was discovered by Zermelo a year before Russell). There were two solutions mooted - conservative & radical, either restrict comprehension so that one cannot formulate the paradox, this is the conservative path taken by Zermelo which ended with the formulation of ZFC which is formalisable as axiom schemas & first-order logic. The radical path is to understand the paradox as an indication that the universe of Sets is too small ( one should compare this with the idea that the set of real numbers is too small - answering the question 'what is the square root of -1' led to complex numbers, an extension of real numbers). In this path, the set of all sets is not a set but a class. That is here there is two types - sets and classes. This is the path taken by Russells theory of Types.
Brouwer, argued against Hilberts formalism and was the instigator of intuitionism where he argued that existence is not enough without justification. This led to intuitionistic logic where the law of the excluded middle is disallowed. Both type theory & intuitionistic logic led something of a underground existence being overshadowed by ZFC and classical logic respectively. However they found a new home in computer science, and then in catgeory theory.
Because Category theory constructs a category of all objects satisfying a predicate - it would need to use classes which aren't available in ZFC. So already we see it break out of mainstream foundations. Theorists distinguish between small & large categories - that is those categories which have a set or class of objects respectively.
It was Lawvere that had the radical idea of using category theory itself as a possible new foundations for mathematics. This resulted in eventually what is called ETCS (the Elementary Theory of the Category of Sets). This is essentially ZFC but formulated in the spirit of category theory. Like ZFC, it is fully formalisable as a first order theory as explained here. Trimble explains the differences between ZFC & ETCS more fully here.
ETCS is better seen as a specialisation of the notion of a Topos. In fact, ETCS is a topos equipped with a natural numbers object, that is well-pointed and has choice. Toposes are seen as a more natural context, since this is the most appropriate level at which the associated type theory & logic (which is intuitionistic and higher-order), and geometric side to them too - the theory of sheaves are seen.
Another piece of evidence that Toposes are a natural context is the remark made by Bentham & Doets in HOL
Unlike ﬁrst-order logic and some of its less baroque extensions, second and higher-order logic have no coherent well-established theory; the existent material consisting merely of scattered remarks quite diverse with respect
to character and origin
But this isn't true of topoi in its geometric incarnation an extensive, sophisticated & forbidding theory has been built by Grothendieck & his co-workers in Algebraic Geometry.
Philosophically this is leading to the idea of a 'multiverse' of set theories in which ZFC is a distinguished one. A theory is interpreted in a topos. Compare this with model theory where one interprets a theory in a model which has set semantics.
All this so far is category theory done in dimension 1. In higher dimensions other features appear. In infinite dimensions homology & cohomology have a natural context within which hundreds of such theories are specialised versions. Now when one looks at an infinite topos its internal logic is homotopy type theory which has been identified as per-lofs intuitionistic type theory. A concerted effort is under way led by Voedvodsky & Awodey to understand this in a foundational manner.
I think one needs to get away from the idea that not having correct foundations means that all of mathematics suffers. Obviously Newton for example was able to reason with calculus even though his theory wasn't foundationally established, similarly with Euclids plane geometry. In a sense different areas of mathematics have their own ontology, their own way of making meaning. Hilberts programme was a reductionist one which aimed at a very pure single ontological level.
Historically mathematics has had a symbiotic relationship with physics - ideas of algebra & calculus have mutually impacted both. But, consider that set theory despite its foundational aspect has had no contact with Physics. This, despite appearances, argues against taking set theory as foundational. This also goes for set theory's relationship with other areas of mathematics inside mathematics proper. Category theory has impacted physics, for example the idea of a TQFT is most elegantly formulated as a functor from the category of n-corbordisms. Another example is higher gauge theory which is a generalisation of ordinary gauge theory, important precisly because it is the language within which the Yang-Mills Theories & EM are written, and with a little cunning Gravity.
Perhaps as a coda its worth enunciating a couple of ideas in Category theory which I find philosophically interesting. One can think of it as a loosening or expansion of Liebnizs prinicple of indiscernibles. If two objects have all the same properties they must be identical.
The first is what is known as universal properties. This is a little badly named, they're better thought of as characteristic properties. Instead of asking specifically for an object one specifies what properties it must satisfy to characterise the object uniquely. This means that any such object is actually not unique - it is unique upto isomorphism. In categorical thinking this is the best that one can do. One can think of this as an loosening - here they're not, they're isomorphic. This means for example the number one is not unique, there are many of them but they are all isomorphic to one another.
But in fact there is more, if two objects are not the same, but isomorphic, they may be isomorphic in more than one way - one can then compare these two isomorphisms, and so on. This leads to an infinite progression. In infinite-dimensional category theory this very expanded idea of equality is the one used.