The difference is simple, non-existent objects don't exist whereas abstract objects might.
For example, round squares are non-existent because they are self-contradictory. The golden mountain is non-existent because it is not actual. Fictional entities are non-existent because fiction isn't literal truth.
Non-existent objects are not (at least, needn't be) abstract. The golden mountain really is made of gold, it is concrete and not abstract--- it just doesn't exist. Likewise, there is a detective who really lived at 221b Baker street, he just didn't exist.
Meinongianism and the theory of non-existent objects has not gotten wide acceptance. Most philosophers who advance some sort of Meinongianism these days try to accomplish the same as the distinction between exist/not-exist with some other distinction(s) like abstract/concrete or actual/possible/impossible.
See this article, especially section 5.4 for more discussion of this.