Also, there's no real need to be defensive. If I'm out on my own or with my cycling pal and we get burnt off by a group of faster riders, as we frequently do, we normally exchange a friendly "hello". They don't shout "eat our dust, LOSERS".
Which reminds me a of a long steady climb this summer when we were passed effortlessly by a group of guys in club kit. Shortly later we re-passed them having a wee stop. Then they came past again. Then we re-passed them changing a tube. And we won the climb! Ha! What a bunch of weeing, deflating, losers! Then we turned off before they could catch us again.
Mind you, even if you - like me - don't harbour any competitive urge when cycling, being a bit fitter does enable you to enjoy your tootling around for a bit longer.