I was never that bothered about 'passengers'. They're usually hanging on for grim death, so would just slow down the group if they did a turn.
Everyone has a 'natural' speed, and some can only go faster by wheel sucking on the flat. What often happens is that larger riders tow a group along to the bottom of a hill, then the lighter riders shoot off. You sometimes get to read about that in accounts. Usually in a reference to the writer's great climbing skills. 'It was a relief to leave the boredom of the flatlands', or a variant of that. It's especially boring for them when they've been watching the back wheel of a tandem for 20 or 30 miles.
The hills between Edinburgh and Brampton can be interesting. The 'climbers' attempt to shoot off, but the wind holds them back. It confuses them into pushing too big a gear, and they get achilles and knee problems. The need to remain 'aero' also causes saddle sores.
In reading accounts it's useful to look at elements of overlap. It's also useful to work out 'outliers'. The rider might have specific dietary needs, be more fastidious than average, inexperienced, or too experienced.