There was a time when lower caste was not allowed inside temples. This has changed since then. However, the absurdity that some one who believes in Hinduism (What other reason can you attribute to a visitor to a temple ?), is not allowed inside the temple just because of his parents reliogion is absurd.
I am married to a chinese national and has been to many temples in Kerala with my kids. I intend to continue the same. That is the only way I could translate my reliogion to my kids. If some one wants to clean something up after I have left the premise, be my guest. At least you will keep the place clean for a difference!!!