Agreed. It's fundamentally discriminatory, and arguably unsafe. And on busy services (those Sunday evening Voyagers are a prime example), there simply isn't room to load and unload your bike without first evicting several passengers, a couple of Luggages and assorted bags of coffee-dripping rubbish from the surrounding area.
Even when it's not rammed, I often have to use the Teacher Voice to persuade people to move so I can un-dangle my bike and re-attach panniers before arriving at the station, rather than getting in everyone's way trying to do it in the limited dwell time available.