It's a covenant. I had a house in Coventry that had a number or restrictive covenants; no caravans on driveways - to prevent travellers siting their vans as the estate was built on former common land (this, in the Midlands, which decamps in it's 'vans en-masse to the southwest come summer), only doctors, dentists or accountants could use them as business premises, etc. All were ignored and nothing ever came of them.
I think that as long as you honour the spirit of the restriction - i.e. it's not a second home / property to let there will be no problems. After all, if you did work locally, but had to change jobs, you'd not be forced to sell.