I considered this very same question a number of years ago. I was looking for someplace to retire. My criteria were; somewhere warm, with a low cost-of-living, that spoke English, and where I could keep my guns.
I looked at Belize, but could never find their policy on private firearms ownership. According to the "Freedom Index" they rank at 30% in that department. While it is hard to get a good understanding from the table, it is my experience that if you have money, more options are usually available to you.
If you retire to a third world country, you will be wealthier than 90% of the residents, and so your standard of living will be higher than the average. If you keep your money in a US bank, and make periodic draws in US dollars, you don't have to worry about hyperinflation in whatever economy you settle. But, since you are one of the “rich” people in the country, you will probably be taxed at a VERY high level.
If you keep your US citizenship, you may be restricted from owning property, owning firearms, voting, or owning a business. You may be able to establish dual citizenship, but some countries don't allow that. If you don't become a citizen of the new country, you will usually need to get a visa that will have to be renewed periodically, and can be denied for any reason.
Finally, you have the greatest "freedom" in a country without a functioning government. But who would choose to live in anarchy?
Oh, and thanks for posting the link to the "Freedom Index". Very informative.