Not sure what to think of this. To me, it's a well-used expression that does not imply that you actually smoke crack. What other expressions should we not speak in case they literally might offend someone?
That expression already indicates an ad-hominem rebuttal, and ad-hominems can offend by definition ("against the person"). By using a higher DH-level (src: http://www.paulgraham.com/disagree.html ) in argumentation, you can avoid a lot of problems and have more meaningful discussions.
