Coopting terms (and stereotypes) can be a defiant way of claiming and enforcing one's identity as a member of an oppressed group.

Examples of terms that are used in context by members of a group, that when used by an outsider are/can be hateful (i cringe to do this): fag, dyke (and variations: bulldyke, bulldagger...), queer, queen, fairy, nigger, bitch, sometimes even girl, maybe geek, i can't do this any more, you get the point...

The reason that it's not okay for you, as a non-member of a group, to use this term, is that you probably can't understand fully the connotative meaning that it has to the person you're applying it to. And also, there's the issue of a power differential (whether historical or present, these things echo in the psyche). So you're not really using the same word they are.