Gee, I thought this problem was settled by Jefferson, and the Bill of Rights. Yes, it's in the Federalist Papers, but not technically in the Constitution. However:
The founders of the US Constitution put this in place because they saw that every time a European government changed, the State religion changed, and, as a result, there was widespread warfare and redistribution of property. And yes, White Men were forced to abide by whichever state religion took power.
They solved the problem by stating that religion was the sole right of the human Individual, never the State. They did this with speech, the use of printing presses, and the right to retain ultimate coercive force over a run-away government (bear arms.) They brought God into the unalienable rights concept so no state or power could legitimately take these rights away. (John Locke made manifest.)
In the Federal Democratic Republic of the United States of America, the government cannot establish a religion anywhere in the country. Likewise, it cannot restrict the expression of religion anywhere in the country.
Me, I'm an agnostic. Leave the historical Ten Commandments artwork around. But dump the Marxists. Marxists don't tolerate other religions, so I don't tolerate them.