On the stem won't interfere with a bag if you have it on the *back* of the stem, facing you. It makes the folded bike wider by the width of the cage, but doesn't actually interfere with the fold. It's also very convenient to drink from.
The other place is on the main tube, just behind the bend so the saddle clears it. However, most bottles dribble when mounted nearly-horizontal.
I have braze-ons for both positions, but you can use Elite strap-on bosses. The trouble with the latter is that they tend to ping off when a cage isn't actually fitted, so drill a couple of holes in an alu strip and use it as a blanking plate.