Intention in offense

If we offend someone with our words, does our intention carry weight? Or on the contrary, does the offended person's feeling take precedence? This question arises particularly in the case of racist, sexist or homophobic remarks. It sheds light on the misunderstanding of the notion of ordinary sexism and structural racism.