Even Islamic countries (e.g Saudi Arabia) doesn't allow other religion people to visit their shrines/mosques. Saudi Arabia doesn't even allow practicing their religion (if other than islam) , doesn't allow their holly scriptures/books to be kept with them.
Are you going to appose it ? But if some temple in some province of Kerla in India doesnn't allow it, you have problems.
