Some use this phrase to imply that another person's preferences are peculiar to them, perhaps low-class or stupid. I use it merely to get the point across that I don't have a taste for the item in question (say, asparagus on toast, which my next-door neighbor in Lambda Hall used to eat). No mockery of a person's intelligence or upbringing intended.