I can conceive of an infinite past with a beginning. I can in fact represent this idea by a simple diagram, part analogical, part symbolic. So, to me, this idea is a logical possibility.
I initially thought that nearly everyone should be able to do the same. Apparently, I was wrong. Many people object to this idea, vehemently, on the ground that the ordinary, conventional notion of an infinite past is that of a past which is infinite precisely because it has no beginning.
So, as the argument goes, the notion of an infinite past with a beginning would be a contradiction in terms, and this even though, unlike for example "bachelor", there is no dictionary definition of "infinite past", and there is therefore no dictionary definition of an infinite past as having no beginning.
As I understand it, our initial notion of the infinite came from our sense that time is going to continue and that, therefore, it is literally not finished, i.e. in-finite, or "not complete" as some people like to put it.
Still, since more than a century ago now, mathematicians have learnt to deal with the notion of actual infinite, i.e. the notion of an infinite that would be complete. This is not necessarily the same idea as that of an infinite with a limit, though.
As I understand it, the idea of an actual infinite came as a consequence of assuming the existence of a set containing an infinite number of elements. The number of elements is infinite but the set itself contains all of them and so is an "actual" infinite. This in itself doesn't imply that the set contains a greatest or smallest element but the set is thought of as containing the entirety of an infinity of elements, which seems to imply at least that the set is indeed a "complete", or an actual, infinity.
However, the interval of Real numbers [0, 1], for example, is conceived of as an actual infinite since, like actually infinite sets, it is conceived of as a definite entity composed of an infinity of points. It also has a "beginning" and an "end". Thus, as conceived, it is an infinite collection of points with an end and with a beginning. Where's the contradiction?
And I also think of [0, 1] as commensurate to an infinite past with a beginning, or even an infinite time with both a beginning and an end. This could be easily formalised.
The interval of Real numbers [0, 1] is only one possible example. We could easily imagine any number of different species of infinite pasts with a beginning. For example, an infinite past with two beginnings, or with two or even an infinity of beginnings (and still just one present time). There is in effect an infinity of possibilities in this respect. So, something broadly like [0, 1] is merely the easy token example.
Something conceived of as the past, also as being an actual infinity of moments and has having a beginning and the present as an end is in effect an infinite past with a beginning and may therefore legitimately be called, and indeed should best be called, "an infinite past with a beginning".
So, how would it be necessarily illogical to think of the past as both being an actual infinity of moments and an infinity with a beginning?
Or why would it be somehow necessary that if the past is an actual infinity of moments, it has no beginning?
EDIT: By time, I mean the ordinary sense of a continuum in which events occur in irreversible succession from the past through the present to the future.
2nd EDIT Nov. 30, 2022: It is clearly not true that the standard meaning of "infinite past" is that "for any time x, there exists another time y such that y precedes x" as Adam Sharpe claims in his answer. This notion is probably that used by scientists and mathematicians, possibly some or many philosophers, but this is not what most people have in mind and so not the standard notion.
The standard notion of an infinite past is instead that of a past that contains an infinity of moments, where the duration of each moment is not zero.
Given this, I still don't see any contradiction between the idea of an infinite past and the notion of beginning in time.