That's fairly normal. The screen on these is a bit of a compromise, and works well in darkness (with backlight) and bright sunlight (of the kind we don't really get a lot of in winter). With the backlight off, it's quite sensitive to the viewing angle, and reflections, so fiddling with the mount may help, but if it's gloomy you probably just have to use the backlight - though finding a zoom level that keeps the display from being too cluttered can help. The reflective display is significantly better in bright conditions than the transmissive display typical on phones etc, and allows for less backlight use overall.
Best approach to conserve battery is to set the backlight to time out after a minute or so, and either wake the screen with a short nudge of the joystick (which shouldn't be enough to put it into the select-a-point-on-the-map mode, but watch out for that happening) when you need to check it, or use route navigation so it lights up automatically at junctions/waypoints.
