One-in-four southerners were directly involved with slavery, whether by being slaveholders themselves or by being in the market for the goods produced by the slaves on the plantations. This made it economically in the south's interest to want to hold on to slaves and their labor.
