No man, if you're a minority then it hard. Mob mentality means that one guy who starts shit will have a whole crowd will carry that forward. You're right, some fans are great like Arsenal because there are a lot of black fans, love Blackburn fans and generally smaller clubs. Liverpool/Everton and Man United are among the worst, low level like Millwall and Leeds.
I was working the game where West Ham were 3 up against Spurs but then you came back to win with Berbatov in the team. At least 10 stewards were attacked that game, police also got hurt. Football at it's best and worst.