When I started riding, I carried stuff on my back, but, for obvious reasons, that wasn't sensible for me, so I very quickly got a rack and cheap panniers. I regretted the latter, and very quickly stretched the budget to a pair of Carradice Overlanders, the left of which got a lot more use.
Actually, I'm really surprised that so many people carry their load on their backs instead of their bikes. I used to get a lot of stick from 'proper' cyclists - commuters, tourers and fast lads - back in the day, and, despite massive improvements in cycle-specific design, I can't imagine it's a good idea still.