Sightseeing, it depends how far you want to ride, if the weather is good a ride up Portsdown Hill is worth it for the views, there is a lay-by opposite the Churchillian pub which is home to the famous Mick's Burger van (home of the Monster Burger and good for any variety of fried thing in a bap and tea).
If you want to go a bit further afield head up to Broad Halfpenny Down (opposite the Bat and Ball, sort of in Hambledon) and see the former home of English cricket, then on to Butser Hill (the second highest hill in Hampshire) followed by a pleasant ride back through the lanes via Old Winchester Hill (22nd) before testing your brakes on Portsdown Hill (a measly 26th).
If the weather is against you then I recommend a day at the dockyard mooching around the old boats but do mind your head.
The Old Customs House is in Gunwharf Quays and you may be pursued by a security guard onna Segway if you ride your bike in the wrong manner etc etc and, as Johnny Faro says, there are a few granite sets to negotiate, not to mention the middle aged, moneyed types shambling around the area.
To get to the pub you need to go into Gunwharf through the pedestrian only archway just to the left of the road access to the carpark, walk/ride through the arch, dodge the Segways and drunks and head in a straight line towards the harbour, The Old Customs House is on your right about halfway down.