Actualy while general helpful, the size especialy as demonstrated by that picture above of various rounds can be misleading.
Lets take the .45 ACP vs the 10mm (not pictured). The 10mm is a smaller cartridge, with a smaller bullet, and smaller brass. However the foot pounds of energy delivered by that 10mm load is greater, around 650 foot pounds average. The larger .45 ACP averages around 500 foot pounds.
This is because the .45 must remain at much lower operating pressures than the 10mm.
However simply measuring weighing and comparing the 2 one would assume the .45 was more powerful not knowing any better.
There is also other factors besides power. The harder a bullet the more it will penetrate, the softer the less it will. Shape also changes this. A sharpened point round of the same material will go much deeper than a hollowpoint which is acts like a cup forcing expansion increasing resistance and surface area and slowing it more rapidly (and transfering its energy.)
Then take Owen's order 5.7x28 < 9mm < 5.56 < 7.62.
The 5.7.28 will go through things that stop the 9mm cold, yet both have similar power and the 9mm can do more damage.
This is because even if you took 2 rounds of the same foot pounds, and same shape and material, the one with a smaller frontal surface area will be applying those pounds to a smaller area on the target. The same principle that makes the point of a knife cut through something with a small amount of force that it would take a hammer with a lot of force to go through. The material delivering the energy also matters, however that is complex enough to warrants its on page.
So even though the 5.7x28 and 9mm deliver similar foot pounds of energy, the smaller diameter round delivers it to a smaller surface area. So which is ideal or more "powerful" depends on what its intended use is. Since both would penetrate deeper than necessary on an unarmored human torso the wider of the 2 would do more effective damage, so the 9mm. Yet if soft body armor was in the way like on most battlefields the 5.7x28 would actualy still do some damage while the 9mm would do none.
Also you mentioned the M4 equal to the M16 based on someone's comment. In fact the M4 at medium ranges loses so much energy compared to the M16 that the military round is not fragmenting as it is designed at those lower velocities and its effectiveness is therefore much lower at stopping enemies in the field having only one wound track instead of 2-3. This has been a problem in Iraq/Afghanistan.
So general statements that one barrel size makes little difference, or that one does make a difference is over simplified and not accurate. It depends on the powder type and the round. There is a range of rapidly increased velocity, then there is a range of gradualy increased velocity, and then it can reach a point of velocity loss. One often wants to stay over the rapidly increasing velocity, while stopping short of making full use of the gradual range so as to keep a realistic barrel length.
However it is the powder type and pressure limitations of the firearm which determine how much space is necessary to enter into each range. Generaly speaking pistol and shotgun powders are fast and the rapid increase range is met fairly quickly in them. However rifle powder burns slower to still reach a great velocity while staying within the pressure limitations of the firearm.
Think explosion. If the explosion is very fast the pressure spike is fast, and what is surrounding the explosion has the force applied rapidly, but then no more. If the explosion is a bit slower then force is still being created as the bullet is heading down the barrel. Since the firearm and cartridge can only take so much pressure before exploding themselves, one must remain below that pressure so one reaches maximum velocity with a given barrel length by applying power to the bullet as long as it is in the firearm rather than all at once. One wants to keep the internal pressure close to (while safely under) maximum pressure as long as the bullet is in the barrel. With a short barrel like a pistol that means you get the most from faster. With a longer barrel like a rifle you can get more from slower.
Since manufactered ammo for a given chamber is designed originaly for a specific length barrel in a given firearm, it tends to be optimum around that range. This mean taking a rifle round and using it out of a chopped down barrel will on average greatly decrease the performance.
So there is no simple answer. Higher foot pounds does not mean better performance across the board, bigger caliber does not necessarily mean higher foot pounds, and more case capacity does not always mean faster velocity. These are general concepts that can give one a rough idea, but one needs to know a particular round to be sure. For example: A smaller case capacity in a firearm designed for much higher pressures could use a more powerful powder and actualy propel a bullet to a higher velocity than one with slightly larger case capacity that had to remain and operate at lower pressures.
So the ideal cartridge is one that is as wide as possible while still going the necessary depth into a particular target from a particular distance, expansion is a way of artificialy increasing the diameter of a round. The distance is important because some projectiles keep thier energy better over longer range, while others would have greater terminal performance at closer ranges but would have lost too much energy at longer ranges. Ballistic coeffecient determines this.
The limiting factors are however the recoil and the cartridge size and firearm design. Wider rounds in a pistol with rounds within the grip for example would decrease capacity. A more powerful round round for round may do more damage, but take several seconds between rounds to aim once more, while a less powerful round can allow one to deliver more total energy in the same time span. The total depth and amount of area damaged in a given time span is important too. If the target is a human, humans are only so deep. So after so many inches of penetration you want the energy to be delivered in increased surface area rather than additional penetration. A bear requires more, an elephant even more penetration before your concern is surface area. A person behind objects does likewise.
So different rounds are better for different situations and different targets.