Vertically on the stem seems to be the usual approach for Bromptons, if only for ease of access. I've got a Monkii Clip on mine, which takes a bottle (not necessarily a cycling bidon) in a Monkii Cage. This ends up with the bottle upside-down at a 45 degree angle if you fold the bike, and is solid enough to carry onto a train like that (though the bottle may leak a bit, so I tend to flip it the other way round). Similar fixings exist from other manufacturers eg. Rixen & Kaul Klickfix.
But more generally, bottle cages do work horizontally. It's pretty standard to mount cages on the boom of recumbents, though they're less of a getting-kicked risk than above the main tube of a folder, and normally a few degrees off the horizontal.