Hello,
I can understand your problem you are on the right way ........
I think you should not compare yourself with his past because he did it when you are not in his life after you entered in his life he only loves you......It's not easy to forget past thing but you can try to forgive his past and try to make a new unforgatable memories with him...
All the best!