I'm sorry if that's a regular occurrence. But if it is, take heart that you regularly pass people! It's true, however, that some people say the oddest things. I was once passed in The Lakes by some bloke rehearsing for the Fred Whitton suffer-fest. He was on a carbo-frame and I was on a hire-bike and he said, as he passed, "I wish I had your gears" which was pleasant enough but I had to resist the temptation to retort, "I wish I was your age, instead of 40 years older," which would have been the politer alternative to "You've just passed me, you don't need my ******* gears!" But I still think it's better for my peace of mind and general enjoyment not to rise to it!
All the best.
Peter