Why should we Hindus worry if Muslims/Christians 'blame' us for idol worship ? Why should we care ? Most human lives have been lost in the last 2000 years due to conflicts by Christians/Muslims.
The idol helps us create an image of God. It is easy to focus on something if you have an image in front of your eyes. This is scientifically proven.
Secondly, we Hindus believe that God is omni-present. God takes up more space than this Universe can provide. So essentially everywhere. In a stone, a mountain, a tree, animal, human. The physical artifact provides the medium to reach God. If someone doesnt need the medium, that is fine too.
We dont need anyone to authorize us on the correct method of worship. We have been doing this for 5000 years when most religions were not even established.