Is probability theory (using Kolmogorov's axioms) an extension of the propositional calculus, or an extension of first order logic?
Probability theory can be understood as an extension of the propositional calculus, and even of Aristotelean logic, but not of predicate logic in general. To clarify, propositional calculus is basically the truth-functional calculus of 'and' 'or' 'not' and material implication. Aristotelean logic permits quantification in one variable, so that one can capture statements such as "all A's are B's". Predicate logic is a great deal more expressive and permits quantification in any number of variables in a single sentence: for example it allows one to capture the difference between "there is some girl whom every boy loves" and "every boy loves some girl" and prove that the latter entails the former and not vice versa - not something one could do with Aristotelean logic.
If one uses an epistemic interpretation of probability, then one may speak of the probability of a proposition being true. Probability calculus provides a way to deal with ands, ors and nots that is compatible with the propositional calculus, but it does not allow one to peek inside an atomic proposition and say that individual components of it are more or less probable. If a proposition contains multiple quantifiers, there is no completely general way of accounting for how the probability of the expressions within it relate to the probability of the proposition as a whole. Some progress has been made into what is called probability logic and it remains an active area of research. Jon Williamson has written about it in the paper "Probability Logic" in "Handbook of the Logic of Argument and Inference" (Elsevier, 2002) and in his book "In Defence of Objective Bayesianism" (Oxford, 2010).