Leaving may be exactly the wrong thing to do. By leaving, you surrender the congregation to the antigunners by default. Check around to see how the other congregants feel about guns. You might be surprised that the consensus does not track the feelings of the minister. Then the minister can be reined in, or replaced.
What we're seeing is a pervasive polarization in the country. People are choosing up sides, not only about guns, but about the whole constellation of ideological issues. Then they select churches, schools, neighborhoods, and even whole states, that reflect their personal values. This is a terrible trend. If you want a civil war, this is how you get a civil war. Instead, let's try to build bridges.
Build bridges? For this you go to the deacons/elder of the church and demand clarification and/or removal. Depending on what they do you stay or walk. Not sure how this ends up in some civil war.