That there were no constitutionally protected rights was the main objection of the anti-federalists that resulted in the promise of a bill of rights being amended in exchange for ratification.
Madison argued that there was no need for a bill of rights to protect the rights of individuals, because the authority of the federal government under the Constitution did not touch the individual--it only touched the states so the protection of individual rights was manifested in the bills of rights in various state constitutions. But the anti-federalists feared the new fed. govt. would in the future seek to extend its power (as it has indeed done) and insisted that certain individual rights must be specifically protected from such federal overreach. Such protection was the price of ratification.
Madison drafted a Bill of Rights to be presented to the new Congress once the Constitution was ratified. Madison's draft was went through several modifications by both houses, and eventually resulted in the presentation and ratification of the first 10 amendments as the Bill of Rights.
But make no mistake, the intent and purpose of the Bill of Rights was and is to protect certain rights of the People, both individually and collectively, from infringement by the Federal government. The 14th Amendment extends this protection to include state governments as well.
Yes, the government can be granted the power to infringe or restrict these rights, and for rights not specifically protected from such restriction, the government already has such a grant in several places in the Constitution. It is against this grant of power that the Bill of Rights affords protection. All that is necessary to give the govt. power over these rights is to remove (by repeal) the amendment in the Bill of Rights restricting the power it already has.