Thats were the polity and politics change. You might be elected due to aprty and you will have to govern based on the broad principles of the party. When you take the oath on the constitution you are member og a LOm Sabha who should be doing things with natyional interest in mind. Being a speaker its the tradition and contitutional duty to be above party politics. Then the party nor individuals matter.