Complicated situation involving friend, her ex, etc :/?

So my friend, her ex and I used to be best friends like 3 years ago. Him and I sort of always had an attraction though, even when they dated. Awhile after they broke up, we had a tiny thing but never did more than make out and finger. The female friend found out and hated me for it for the longest time and somehow thinks we had sex but it never happened. (He used to ask me to when they were still dating but I always declined, yet I'm made out to be the bad person!) Anyway we settled our issues and have been friends for awhile, but not close as before at all. He is in the marines and has been gone for two years in Japan, and just now coming to visit for a month. I hung out with him the other night at a mutual friend's get together. And his ex/my ex best friend found out and is being a ***** about it. For one I have a serious boyfriend of over 3 years who I love and I would never cheat on. And two it's not like i was alone with the guy! I invited him over to the place but meant nothing by it, i just wanted to see an old friend who has been gone for ages. This girl is a nut case and is obsessed with him and he wants nothing to do with her. Plus she has had sex with literally half of his friends since he went overseas and yet it's the end of the world if i just hang with him? I guess I just really need to vent. But I wanna know what I could possibly do in this situation. I want to hang with him again but can't stand this ridiculous drama!! Am I in the wrong here? What pisses me off more is she starts getting our mutual friends involved and make them side with her so i get stressed about it all even more. Sorry this is so long and thanks for reading and hopefully helping :/