Amanda, all I can say is, there are no hard or fast rules in something like this. It is not about what he knew or you did not tell him. It is about unconditional love and unfortunately men, or women does not like to hear about past lovers, before or after the marriage.
Many people have asked if they should tell a partner about the past and almost every time, depending on some real special circumstances I told them no. You as a new Christian should know that the past is the past and better left alone especially if you started a new life in Christ. The past has no relevance in love.
When a couple start loving each other, it is a whole new life, but as soon as you divulge unsavory secrets from the past, you bring uncertainty into that new relationship because the human mind is unfortunately a very devious tool in conjuring up all kinds of vivid mental pictures and with that comes thoughts such as, "Will he or she do stuff like that again... How can I trust him or her?" It is also called jealousy (a tool Satan likes to use) and what makes it worse is when sometimes in the heat of the moment you throw something like these secrets in his or her face.
So before you listen to well meaning opinions, ask yourself this question and be very honest about it, "Is my marriage worth fighting for, and do I love him enough to really ask his forgiveness in telling him about the secrets and to communicate my unconditional love for him?" You also have to get him to open up to you and tell you about his fears and then you both have to decide if your love for each other is stronger than the past, and then you both have to decide to close the book on it, but... also to communicate openly and calmly if ever it comes to mind again.
A marriage is a holy contract before God, and each and every one of us has his or her baggage, but when you said for better or worse, you not only told each other that, you also told God that your commitment is total before Him.
There will be many ups and downs in your marriage but if your commitment to each other is complete, you will be able to weather all storms. Believe me because I've been at it for more than 42 years, and many a time I had to eat humble pie, whether I knew I was right or not, because I knew it was worth it.
I hope this will help you and if you wish to communicate some more, please feel free to do so and May God bless your marriage.