If the temple management is of the view of not permitting non hindus, well then that's their decision and others should respect it. No one should be offended by it. Its only those people who are complaining who themselves never go to their own places of worship.