Two main things some to mind.
First. Many Audax riders do not come from a back ground where they have learnt the method and etiquette of group riding. For instance, they will sit on another riders wheel and contribute nothing. When the rider in the front starts to fade a bit, and could do with a wheel to follow, the sit in sprinter wizzes past and breaks up the group, usually to end up 100 yards up the road. Eventually they fade and get caught again, and repeat the process. If you try and explain to the how they could work with the group, they get stroppy.
Second. Very often you go through an event naturally joining up with one or more riders, but often the group evolves and sometimes completely changes.