It's not the actual legal situation which is the problem - it's the potential for correspondence and discussion and explanation etc etc etc, with multiple people. Same as when you photograph the houses of Parliament and then have to explain yourself to a dim plod. It ends up amounting to the same thing - in other words, something to be avoided.
We don't obtain permissions for Arrivee cover shots - but somehow, people feel they have to be more sensitive about the web ...