Walk around, the quiltwork of neighbourhoods is unique. NYC is my home from home, I've been back-and-forth since the mid-90s (and in the mid-90s, there were still plenty of neighbourhoods in Manhattan you didn't want to visit, now it's pretty much all very safe). The Bronx back then? Fuggeddaboudit. It's a big city, you won't see it all, but it has a character and vibe that isn't represented anywhere else. It's America yet it isn't.
Cycling is a lot better these days, but unless you're familiar with the stop-start nature of NYC and city cycling in general it can be a bit daunting. There are a growing number of cycle lanes, but don't expect not to share them with the occasional errant taxi. Marvel at the delivery boys on their clunky (occasional electric) bikes and their total ignorance of traffic direction, signals, and the weather. Getting absolutely anything delivered isn't as exciting in the era of Deliveroo and Uber Eats, but back in the late 90s, it was a serious novelty. The delis and bodegas are a NYC institution in and of themselves. The remixed whoop-whoop of sirens is one of the best soundtracks (provided it's not a NYPD cruiser behind you, in which case, sorry officer I didn't realise it was one way.*)
You can ride and walk up the Hudson (well, the riverside, obvs), there's a park (in various states of completion) all the way from the Battery up to the Cloisters and Fort Tyron and Inwood Park. If you like galleries, go to the Cloisters anyway. You can get down the East River side, they're extending the paths, but it's also a interesting walk, with views of the rapidly gentrifying Queens and Brooklyn watersides.
Cheap, fun escapades include – of course and as mentioned – the Staten Island ferry. SI has some surprisingly pleasant beaches that really are a world away from NYC (and a subway ride from the ferry terminal). Or if you haven't got a towel and don't want to turn around immediately without leaving the ferry terminal, Snug Island Harbour Cultural Centre and Botanic Garden is worth the fifteen-or-so minute walk.
The Roosevelt Island tramway (a cable) car with get you an interesting view for the the price of a subway ride. Roosevelt Island itself is now most upscale apartments though, in the late 90s it was still a bit spooky.
All the bridges have pedestrian and cycleways. The Brooklyn Bridge is the most famous and they do a great job of fixing it up after the frequent monster attack and alien invasions but does get tourist clogged. The Manhattan Bridge isn't as pretty put dumps you in the delightfully named Dumbo (Down Under the Manhattan Bridge Overpass). You can count all the hipster commuters crossing the Williamsburg Bridge.
Central Park is the definitive park, of course, but the trek through Brooklyn to Prospect Park, it's venerable and lesser known sister is worth the effort. Strolling through Cobble Hill and then SW to Prospect Park which abuts the park is a bit of NYC that most people don't see.
Lots, lots more besides the guide book stuff, but I'll stop before I bore people. It is worth getting out of Manhattan and seeing more of NYC, there's five boroughs and they all have something to see (Yankee Stadium in the Bronx, Flushing Meadows/Kissena Park in Queens, Coney Island in Brooklyn etc.). The subway and buses are cheap ($2.75 a ride, anywhere in NYC, you need a Metrocard which the machines will vend and top-up).
*shamelessly played my I'm Foreign card to get out of that ticket.
