There are some fine botanical gardens in Neston. Also, a walk up Frodsham Hill is well worth it. It's about 500 feet up and you get a very good view across the Mersey to Liverpool.
Ness Garden are closed currently, opening to members fairly shortly but no date as yet for public access.
Frodsham Hill is a long way from Birkenhead.
Not much open public building wise yet in Birkenhead, don't visit the town centre it's just nasty, though Hamilton Square is one of the finest Georgian squares outside London.
A walk around the Lever Factory village of of Port Sunlight is worth it, also the nearby Price's candles estate at Bromborough Pool, then why not the Lever country estate housing at Thornton Hough, first two by Merseysrail (Port Sunlight Station) last one by bus.
You'd be far better getting the Merseyrail train to Liverpool for the waterfront, and all the 'recent' history around, or go to Chester if you prefer much older history.
Day Saver ticket £5.55 for travel after 9:30, good for Bus or Train (trains are very bike friendly, for the bike you don't have with you...)
Day Saver ticket covers the ferry service too but only as it is running as commuter service between Liverpool - Seacombe.