I don't think Whiteys should say it at all.
African Americans can say it all they want in my opinion, like Tottenham with the "Yids".
It was a diss, but many years later they embraced it and now call themselves that, which is what black culture does now.