Are ALL the confederate flags racist?

I don't think the flag is racist but history has painted the civil war in very simplistic tones which has made the confederate flag a symbol of racism. Consequently, I can understand how some view the flag, respect that and would never walk around with one or have one in my house, etc. Being from GA, I see it as a symbol of the fight for states rights and not as a symbol of oppression.