1. Get a Garmin. Even if you are a natural lone wolf (aka Linux user) rather than a sheep (aka Win user) I would still say get a Garmin, because other makes tend to arrive set up in a country-specific way, whereas Garmins in combination with free OpenStreet mapping work equally well all over the cycleable world - as a GPS jolly well should.
Want maps.
Have been known to throw misbehaving IT kit at walls.
Maps on a GPS are always a disappointment. Don't expect too much, in fact don't expect much at all, on a 2x1 screen. Fortunately Garmins are very robust and will stand the odd impact.
2. Make a fundamental choice between Garmin's cycle-specific 'Edge' range, and their various models that are primarily aimed at walkers, notably the Etrex and Oregon models. (Handlebar mounts are available for these types.)
The Edges are small and neat on the bars, and include some training aids, but are very much set up as 'day ride' devices with a limited run time of around 9 hours before they need to be recharged. They can be powered externally for extended rides but of course that totally destroys the 'small and neat' which IMO is the main part of their attractiveness. They are also somewhat slanted towards people who live in the 'connected world'.
The walkers models all run off 2xAAs, easily replacable in moments, and personally I find that makes much more sense. This tends to make them look and feel brick-like, but actually if you place an Etrex 30 and Edge 810 side by side there is not a lot of difference in size and shape. They are to some extent the old-skule option.
3. Other fundamental choices are between 'touch screen' and 'buttons' and between 'big screen' and 'small box'. Garmin has models that fit all these combinations. The Oregon for example is 'big screen' and 'touch screen' - the Etrex 30 which has already been mentioned upthread, is 'small box' and 'buttons', the Edge 810 is 'small box' and 'touch screen'. Bearing in mind that these touch screens are definitely not tactile marvels like the iPhone!