It sounds like both you and your husband had troubles from the start (precipitated by finances?) and it never got better. At this point my guess is that there is alot of mutual recrimination and fighting that the children are exposed to.
Clearly your husband sounds depressed, and in his own weird/twisted way, is asking for some sympathy with the "i'm gonna kill myself" line. Unfortunately, bringing a gun into the house is also a veiled threat. You do have a reason to worry whenever weapons are in the house. However, (as i'm sure you know), if things get too heated, anything can become a weapon.
Having stayed together so long it seems there is a grain in both of you that you want to work things out but never quite figured out how to do it. And now it feels too late? I gather that because on the one hand your telling him to get out, then you find him looking for jobs/women/etc and get upset....
I also believe that you think that once you are divorced there will be a great husband waiting for you out there (like the remarried woman you mentioned).
It could be. There are never any easy answers.
However, one thing is clear.
Both of you need to make a decision as to what direction your lives are going to take in the next couple of years. Do you see yourself married in 5 years with the same man you are living with now? Does he?
IMHO it's never too late for any relationship. However, you need to decide. What will you do? Discuss it. Talk about it. Make it a priority what you will do. If you find that the kernel you still have for him is viable, then make that decision. If not, then talk about it and make the necessary plans.
Whatever you do, don't keep ignoring the future.
Either way I would suggest you keep in contact with rational people that can help you make decisions. Everybody's situation is unique and you are ultimately the best qualified to make decisions for yourself.