If you accept that objective reality exists (and reality follows some consistent set of rules), then there would be a single objectively correct probability. Different probability models are closer to and further from this true probability.
So, to argue against you "feeling sure" about some probability (or other probability models), one could compare the demonstrated reliability of different probability models, and present a model that's demonstrated to be more reliable.
Bayesian doesn't necessarily mean purely an arbitrary guess. Ideally, the probabilities we use are based on things we know about reality, to get us as close as possible to the true probability.
It's "subjectivity vs subjectivity" to a similar degree that different distance measurements between countries is subjective. With different levels of knowledge, you can come up with better or worse measurements, but ultimately countries are some actual distance from each other, so any estimation would be more or less wrong. You could also measure the distance from different points in each country, similar to how you could measure the probability of something under different conditions (e.g. flipping a coin may have different probabilities based on how it's flipped and where and how it lands).
* If one were to flip a roughly-symmetric coin a few thousand times, you're highly likely to get heads around 50% of the time. Very few people would disagree with this, and plenty of people have tested this. So it's reasonably assumed that the objective probability of such an event is 0.5.
Some events may not be as easily repeatable. But we don't actually need to ever know what the probability is for such a probability to exist.
From another perspective, if you hold that determinism is true (and possibly even if you don't), one could argue the "true probability" of any given outcome of a single instance of an event (e.g. getting heads when flipping a coin once) would in fact be either 1 or 0, as in: it happens or it doesn't happen. But we often can't know this ahead of time with a particularly high degree of confidence, so we come up with the best estimation of the probability that we can, instead.
In this case, if you take some method of estimating probability, and you apply it to many events, the method would have a certain reliability for predicting the outcomes of those events. If the method is perfect, it would predict every outcome is correctly with a probability of either 1 or 0. How far every prediction is from the correct value of 1 or 0 would represent how wrong the model is, objectively speaking. We can estimate how wrong a model objectively is by checking how wrong it is over a certain period of time (there's also a lot to be said about how to estimate this well, and that's what a good portion of statistics is about).