Understand the contents of his writngs first. You are missing the essence of it. The emphasis of his writings are on the purity of the person to enter a temple. "Brahman" by word is not caste but person who is pure in attire, deeds, conduct etc. This was the code of the conduct in ancient days and has been followed by priests now. If you can recollect women having their cycles are not allowed to enter, touch any thing in the puja room as the have an impurity of passing of blood and fluids.
There are scientific reasoning for the most of the Hindu beliefs and practices. But for an westerner they may look as blind beliefs.