Have you ever used self-depreciating humour in an attempt to deflect insults? If so, then you may be able to understand or appreciate the mindsets of the the people you mention above. Hence why it may be unacceptable for those who were using it as in insult in the first place to continue using it.
I think also that it is only a certain section of the african-american population which uses it, ie, young males.
As for why other groups have not taken up their insults for their own uses, perhaps you could start this trend up in other communities or social groups, DMan...?

---

Freedom of speech? You are free to use any term to any person, DMan. The question is not one of freedom, it is one of acceptability. Remember that society is only how you perceive it, so if you wish to explain to those who you wish to use the word "nigger" or "chigger" to, then feel free to do that too. Perhaps certain sections of the Chinese community do complain about whites. I would not rule that out.