Slavery was one factor that gave rise to the war. Lincoln addressed it in his inaugural address and promised to leave the institution intact. While the majority of men who wore the gray (and butternut) did not own slaves nor cared a wit about them, they were concerned about states' rights and afraid that their lives (as miserable as some may been) was threatened by the North.
By the same token, most of the boys in blue didn't march off to free the slaves. Some certainly did. Robert Gould Shaw and several officers of the 54th Massachusetts were abolitionists. What did that majority fight for then? Why, to preserve the Union (and for the honor of the Nation). Remember the song, Battle Cry Freedom? It goes, "The Union forever, hurrah, boys hurrah! Down with the traitors, up with the Star"[/i]
Don't forget about the Draft Riots, the biggest of which was in NY and protrayed in the movie, Gangs of New York. The rioters didn't want to die for slaves and killed many folks of African descent living there. They even burnt down an orphanage and killed any kids not rescued by the firefighters or police. As a sidenote, the repression of the riot was my favorite scene. The Federal troops had just won a great victory at a small town called Gettysburg and were incensed and saw the rioters as traitors. When they marched north to restore order, they were happy about it (also beats fighting Rebels who can give as good as they receive).
Was seccession wrong? Apparently not. Nothing in the Constitution prohibited it. Recall that Jeff Davis was never tried as a "traitor" for leading the rebellion. The North had another "tricky" issue to contend with too. Remember West Virginia? Well, that was part of the Old Dominion State until McClellan wrested control of it from her. The North then allowed the residents to secede and form a new state. How does this tie in? Well, if a portion of a state could secede, then why not a state from the Union? That was one embarassing issue that no one wanted to raise should Jeff go to trial.
The mistake the South made was firing upon Fort Sumter. They should have starved Anderson out (before being fired upon, Anderson ran out of Hard Tack and his mess sergeants were serving rice and salt pork). He would have left in another 24 hours anyway. Beauregard couldn't wait (and didn't know if another supply attempt was going to be made). Both sides were naive in thinking it would be a quick war. Lincoln called for 75,000 men for 90 days. Everybody believed one battle and it would be all over. 600,000 dead later, opps. Too bad the family feud wasn't resolved diplomatically.
I would like to close with comments about Wirtz and Andersonville. Byron is right in that the South could not deliver food stuff to the PoWs. There were many unnecessary deaths associated with starvation. Wirtz to his credit complained to Conf. General Richard Lee (see Destruction and Reconstruction, p 216, "In this journey through Georgia, at Andersonville, I passed in sight of a large stockade inclosing prisoners of war... a man who said his name was Wirtz... complained of the inadequacy of his guard and of the want of supplies, as the adjacent region was sterile and thinly populated. He also said that the prisoners were suffering from cold, were destitute of blankets, and that he had not the wagons to supply fuel. He showed me duplicates of requisitions and appeals for relief that he had made to different authorities...").
Wirtz was no angel though. Prisoners complained that he robbed them and that his guards were brutal (the original camp guards were soldiers and treated the prisoners with empathy but when they were replaced with 2nd line troops - old men & young boys, the cruelty really got bad). In Writz's defense, the Confederate supply system was horrible. Supplies (including food) did not reach their own armies and rotted in depots. Unlike the North which appointed a railroad man in charge of all railroads, the Southern railroads continued to operate for profit and if the military had to wait by the wayside, so be it. That's one advantage of a centralized federal government over a confederate one. The Confederates couldn't do that without risking become a "federal" government. Finally, the cutting off of PoW exchange didn't help any (blame Grant & Lincoln). However, partial blame must fall on the Confederates themselves since the Union wanted to exchange 1:1. However, the Confederates refused to recognize Afro-Union soldiers and would not exchange them. Since they refused, the Union refused altogether. Unfortunately, men on both sides suffered and languished in prisons because of it.