In my country (Russia), we can freely choose the religion too. However, this is just "on paper", while actually if you did something against Christianity terms you might get sued, and if you're atheist here, most people dislike you, think you're not normal and act aggressively in your side.
This is why I am asking this since I do not really believe in god(s) and want to avoid problems with Japanese citizens when I come to Japan^^.
I think it’s better to behave politely when you visit Japanese shrines, temples, churches, and the Palace of Emperors. In such places, you are expected to behave suitable to that. I recommend you to mimic other Japanese there. I think this is only one point you are better to take care about religions in Japan.