If you could not ride faster than the limit between two controls, you could be in trouble.
Assume you get near the limit on an early control, perhaps because of a p*nct*r* or because the first leg was into the wind. How do you recover the situation?
As long as you do the rest of the ride at more than the minimum average speed you'll be gaining the time back as you go along.
Imagine a 200km event with controls every 50km. Limits of 15kph and 30kph.
Arrive at first control (50km) after 3h19m (1 minute inside the time).
Do the remaining 150km at 15kph and you'll finish right on the time limit (13h19m)
Do the remaining 150km at 20kph and you'll finish with a total riding time of 10h49m
Do the remaining 150km at 25kph and you'll finish with a total riding time of 9h19m.
Do the remaining 150km at 30kph and you'll finish with a total riding time of just 8h19m.
Take the 30kph average (still below the maximum average speed). Add on an hour for controls (if you're fast you can do this) and you're finish the whole thing in 9h19m, faster than I've ever done a 200km Audax, even after being on the edge of time at the first control a quarter of the way through the ride.