There's a car park on the south side of Curzon Street, opposite Millennium Point. Street parking around there will be very limited.
If you're arriving early, then you might be able to get free street parking a bit further south in Digbeth (between Deritend and the ring road), and there are multiple independent wasteground car parks around there where you can pay £3.80 for a day. Don't leave any valuables in the car though, eh.
There are multistorey carparks on Edgbaston Street, including the official one for the Bull Ring shopping centre around there. Beware the Moat Lane car park which operates on market time and could be closed by the time you get back to your car...
I'm not overly familiar with the northern side of B4, so there may be better options depending on where you're heading to.