No friend its not like that, if ur intention is good then u can go over there, no need to ask any muslim friend, no one wil force u to go outside, i had taken many of my hindu friends to mosque, and also i had gone to many hindus temple. We can go each other worship places, just to know each other community people mentaliy, this wil make people secular