For running, I think the best option is a GPS watch. Look at the Garmin Fenix 2. You can load a route to follow, and can display some really basic maps. Plus it has the usual training features, and can connect to a heart rate monitor etc. And it has a low power mode, which can give very long battery life (up to 50 hours).
Or I think the Suunto Ambit is a similar sort of thing. Check DC Rainmaker for reviews.
It would work on a bike, just strap the watch to the handlebars. Though the screen is a bit small, and could be fiddly to press the buttons etc. So something like an Etrex would be better for cycling.
I have done a bit of running just carrying an Etrex in my hand. It works OK, but I find it soon gets annoying, the extra weight is tiring. Attaching it to a rucksack strap would be better. Or just keep it in your bag, and only get it out if you are lost. But then you waste time stopping, plus more likely to get lost in the first place.