If you feel he is just fooling around with you in the name of love, dump him and never look back. He is just using you for his own whims and fancies. Over the years, he has had enough and very soon he would not look at you. Plan a better life, you have the time and there is a good partner somewhere for sure.