See lower caste people were those who did field work working out in the open sun, and hence they got dark skin.
People who did other work like studying vedas and other brainy work were not exposed to fierce sun, hence they got fair.
So in that era, if you would find a fairer person, you would naturally perceive him to be a brahmin and not a field labourer.
This is how fair skin = higher status came into being in India society. No relation to british or mughals, since they are a mere patch on indian history that stretches over 10,000 years, while these came only much much later.
