Actually, the swastika is an ancient and nearly universal symbol for life and renewal, so if that's what you meant by "lollypop heaven", then yeah...
And the Confederate Flag is more a symbol of "states rights" in a federal system than it is of racism. The Confederacy, the American Civil War, and all the symbols associated with it have been misconstrued by revisionist historians. The Confederacy was not formed to promote racism, but rather to present the alternative of rural agrarianism to the aggressive urban commercialism proselytized by the Union.
But, in the end, your point stands. Racism sucks, and people who use symbols for their racist meanings should really stop and think about the power of symbols and the way that their use can be destructive to all parties involved.
That is their way. They usually reserve the most vicious attacks for the best players of colour...