Lets see here...
I wouldnt get back together with him. Chances are, hes just using you for the 'sex' part in the relationship. If he brakes up with you twice... does that mean he'll do it again? Probably. What was his intake when you lost your virginity with him? Was it his first time too? If not, if he'd had 'sex' before, then I would mistake this little thing as him using you for that reason.
If you feel the need to move on with your life (which you should), then move on. There are tons of guys in there for you. The right person for each. And obviously, this guy isnt the right one for you. Yes, you lost your virginity with him, and he'll always be that special person with you, but dont let that fool you how he acts with you.
Go on! Move on! He showed you that he isnt right for you twice already.. why is that letting you fool you?
GBU and I hope you make the right choice with you.
Love,
Russell/Russell07