One reason not to "pop a bullet" into your own head is that it would probably deny you the DMT trip one is supposedly entitled to upon dying: http://en.wikipedia.org/wiki/N,N-Dimethyltryptamine
DMT is the substance suspected for near-death experiences, and if we must go one day, then flying away into the DMT clouds is probably more enjoyable than whatever sensation accompanies the partial destruction of the brain by a bullet.
As for the "point" in an Atheist's existence, I suppose that if religion can give one's life purpose, then there is a good chance that other sources exist which may make your life worth living.
For me personally, being a non-materialist, I find consciousness and existence to be an unfathomable miracle and an endless source of wonder.
EDIT
Here is an interview that I love with Woody Allen about the meaning of life: https://www.youtube.com/watch?v=2MsuqvLIttk
It just seem like a big meaningless thing; now, you can't actually live your life like that; because if you do, you just sit there and why do anything, why get up in the morning and do anything ? So I think it's the job of the artist to try and figure out, why, given this terrible fact... why do you wanna go on living ? What do you care about anything ? If this terrible truth, this meaningless end of everything; and you have to try to figure out knowing that is true, not giving yourself a fake heaven and hell and nonsense, but knowing the worst; figure it out even knowing the worst why it is still worthwile. That's a tough assignment to explain to somebody why it's so terrible and why it's still important to go on. And this is a challenge for artists all the time to try to figure it out.