What is the intuition behind time being a physical dimension? I read the phrase "in cosmology terms, far away means long ago" somewhere and I got to thinking what if time is an emergent property of the interaction between matter and gravity. How can we think of time outside of space and matter? Or does it not need to be? Or am I just confusing matter and physical dimensions?
The intuition behind time being a physical dimension is in the experience of being able to "travel" through it. However, in daily life we only seem to be able to move in one direction and at one speed. In theory, we enter the realm of traveling through time forward, backward, and at different speeds. Thus it has traits of physical dimensions. The phrase, "in cosmology terms, far away means long ago", can at least refer to what we see. Because light takes a specific amount of time to reach us, we are actually seeing an image of something which existed in the past. The farther away it appears to be, the older the image which we see. One way we can think of time outside of space and matter is by thinking of space and matter being inside time. Time does not necessarily need space or matter to exist. If space and matter exist, they do so for a length of time. Matter exists in the physical dimensions. The physical dimensions describe matter. Thus, there are two uses of the term "physical dimensions". The first is the name of a place, the second is a measurement.
There are various theories whereby spacetime (the whole ball of wax) is emergent, e.g., https://arxiv.org/abs/1504.00464 Reading between the lines of your question, I'd guess the best way for you to think about it intuitively is as follows. What's more fundamental: "objects" or "events"? To best interpret time as just another dimension, instantaneous "events" would be the better answer. Then, say, "eating breakfast" and "eating lunch" are two events, separated not only by the distance between the two restaurants you ate at, but by the several hours difference when you ate the meals. In this kind of view, "objects" are just highly-correlated (and kind-of-continuous) sequences of events.
19th Century proponents of Time as a fourth physical dimension included Gustav Fechner (under the pseudonym of Dr. Wises) and C.H. Hinton. The idea was popularised by H.G. Wells in his 1895 novel The Time Machine. Hermann Minkowski and Albert Einstein soon gave it its modern formulation. It is motivated by the idea of a "block Universe", of a mathematical description of spacetime spread out like a four-dimensional map on which we can trace our timelines, in much the same way we can trace a journey on an ordinary map.
Of course, the Minkowski metric of Time's imaginary multiplier and Einstein's shape-shifting relativistic coordinates provide a rather more sophisticated model of the block universe, but from the perspective of Time as a fourth dimension, that is all they do.
As for time, space and matter, the quartet of mass, energy, space and time are welded together by Relativity into a recursive and inseparable whole sometimes referred to as MEST. None makes any existential sense without all the others. For example, as Einstein explained, "Spacetime tells matter how to move, matter tells spacetime how to bend". So having one or more as an emergent property of the remainder is an attractive proposition. Various speculative ideas have been bounced around by theoretical physicists, including wider ideas such as Time as an emergent property of quantum thermodynamic systems (whatever that is) but none has yet to make any headway.
Some nuggets do appear. In string theory there are typically six extra dimensions, of which three, like time, have an imaginary metric. Their imaginary multiplier arises from a quite different principle and they are definitely spatial dimensions. This suggests that Time in the Minkowski metric is not a spatial dimension in the same way, though it offers no help as to what it might be instead.
Then there is the Twistor theory of Roger Penrose, in which time and space are replaced by something more like frequency. Technically twistor space is a Fourier-like transform of objects in ordinary spacetime, analogous to the way a sound frequency spectrum is a Fourier transform of the physical waveform in time. It suggests that space and time are indeed closely related.
So at the moment, we can say that space and time are closely related, but we are not sure how closely or in quite what ways they differ.
Special relativity demonstrates that time and space are related in a fundamental way. Consider a muon created in the upper atmosphere. These particles will travel only about 456 meters (or 2.2 microseconds) before decaying yet survive to the Earth's surface and below. From the viewpoint of an observer on Earth the time dilation effect of special relativity results in time passing more slowly for the muon and it reaches the surface before decaying. For an observer riding along with the muon, the muon's clock is not slowed but the distance to the Earth's surface is foreshortened, and so the distance it must travel (in its reference frame) is less than the 456 meter decay distance. In both cases the muon survives to reach the Earth's surface, but the reason is due to time dialation (as viewed from the Earth reference frame) or distance foreshortening (muon reference frame). This means that time and space are two sides of the same coin, so to speak. It has an analogy in the wave-particle duality of light and matter in which an object (particle of matter or photon of light) exhibit both wave properties (e.g interference) and particle properties (e.g. photoelectric effect). It is not intuitive to regard something as having properties that seem mutually exclusive, but it is nonetheless the nature of our universe. It would not be correct to state that time is independent of physical dimension as theory and experiment demonstrate otherwise.