I tend to be more ruthless with what I carry if the weight is on my back (rucksack, never got on with messenger bags).
With panniers I tend to just accumulate stuff without realising. I emptied mine the other day and found a bunch of extra tools and a full bottle of suncream (probably from when I went on a cycle ride with my 7yo in the summer).
Everyone is different but:
* I'm fine with 10kg on my back for anything up to an hour long commute.
* I sweat enough anyway that having something on my back doesn't make it any worse
* It feels like a smoother nicer ride with no panniers (requires a good rucksack though that doesn't swing about).
That said, I only really do one main cycle commute a week now and I have too much stuff (clothes for 2 days, towel, laptop, lunch for 2 days, coffee mugs, 5-a-side football kit including football(s), astroturf boots, bibs, shower stuff, etc) than I can fit in a rucksack so it tends to be two 20L panniers plus some extra stuff in a rucksack.