Does having your son wear a shirt with a confederate flag on it mean you are raising them to be racist?

Close minded people view the confederate flag as showing racism, that's why in most schools kids can't wear them and in some places they can't be displayed. I'm also from a southern state and my family are sometimes considered "rednecks" but I'm not a racist person and I won't raise my children that way.