You need to do an analysis as to what is meant by a hajj subsidy in reality! Do you know that the ever green Air India goes into profit because of Hajj pilgrims??? So it is that small share of the profit that is offered as a subsidy.
If you want to roll back such a subsidy more than welcome but that should be done on all levels and should apply to hindus as well (such as retrieving security for yatras which cost crores, building of bridges, railway stations for amarnath, kashi etc....)