the minimum stress concentration factor (SCF) you will generate by simply drilling a hole is about three, i.e. the local stresses in the tube will be magnified by a factor of three. The maximum is a lot more than that; very often the edge of the hole is all raggy underneath and this is a perfect spot to start cracks off; in fact it could be said that it is cracked already.
The effects of drilling holes are worst in parts of tubes that see high bending loads (as well as high torsion), so worse than bottle bosses are things like brake cable or dynamo wire holes in the front of the top tube and the down tube. I have lost count of the number of frames with such features that I have seen break.
Having said that, the frame is always likely to break at the worst stress concentration first. In welded aluminium frames this is nearly always at the weld beads even if the frame has lots of nutserts in it. Thus in such frames (and others with inherently high stress concentrations at the joints and plain gauge tubes) putting in a few nutserts arguably doesn't do that much harm; the frame is probably going to break somewhere else first anyway.
Note that it is difficult to make good comparisons between framesets unless you have a fair amount of information. For example a frameset with a '531 ST' tubeset might have completely different tubes in it depending on when it was made and who made it. So you could at one point buy a '531ST' frameset (with nutserts in the down tube and seat tube) from Dawes or a '531 ST' frameset from Raleigh's lightweight unit (no nutserts!) and the weight difference was over 1 lb. That some of the tubes were utterly different would be evidenced by the seat pin dia, up to 1.0mm difference.
The Dawes frames were made using lugs that had the sloppiest fit imaginable, (so there was an insufficient fill of spelter internally, making the lug to all the work) and they tended to break at the lower head lug first anyway.
BTW braze-ons are better but only if they are done well; in the worst case the braze-on will be a high area of residual stress (which may greatly exceed the typical service stresses), together with a stress concentration and a nasty brittle microstructure.
You can mount bottle cages in all kinds of ways; under the saddle, to the handlebars (very retro...) band-on bottle cages, you name it.
cheers