I Told Him About the Number of Men I've Had S3x With, he Always Brings it Up When we Fight

Hello Naijapals,
     I love your site and just want to say thank you for this opportunity to be able to express our inner fears and deep secrets without been condemned. I wanna share my story with you but I will like you to counsel me before you post it.
     My guy and I have been dating for two years, but before we started, he made me confess to something I did which was sleeping with a married man. its something I'm not proud of, I wasn't even dating the man, he was a friend who wanted sex from and in a moment of weakness, it happened. My guy (we were just friends then) suspected and confronted me. After I admitted, we started a relationship, although I told him we didn't have to if he couldn't get past it but he told me he not to worry. He also made me tell him the number of guys I have been with. At the beginning, it was rough, he brought it up almost all the time, he go angry anytime he saw the man but with time he let go (or thought so)
      Now it's been two years, he still brings it up indirectly or directly. Anytime he couldn't reach, he starts calling me names. He always makes me feel like a slutwho had to change because of him. He says things like, I dont have shame or I've started this my lifestyle. He doesn't trust me. I know I did something wrong and I have made mistakes, but I know I am a good person who always fell in the hands of wrong people. I was innocent and naive. I trusted too much, I just wanted to be loved.
      Both Families are aware of this relationship and we are talking marriage but I don't want things to continue this way.  have talked to him severally. I don't know what to do. I have invested my time and emotion in this relationship, he has got other issues like the way he talks to me atimes, the way he seems not show how much he appreciates me and the way he is so prudent when it comes to spending on me but I don't let this things bother me. I always try to make things work. he is a good man and he doesn't cheat but I don't think I can handle this lack of trust and insecurity. Help me please. I am confused.