Kevin,
I think you can discount the Swansea/Abergavenny option. That would mainly be the A465 - 'Heads of the Valleys'. The eastern parts beyond Hirwaun would have some nasty, fast traffic on it, and I don't think there are parallel, quiet roads.
I think your broad choices are:
1. Swansea to Neath Valley to Hirwaun (there are pleasant, quiet roads parallel to the A465 here). Then down the valleys - Aberdare, Mountain Ash, Nelson, Rogerstone. Most of that is the the start of the Marches and Mersey roads 600 and it's quite fast, with a hardly noticeable climb of the Neath valley, a slightly stiffer climb to Hirwaun, then a long, long gradual descent pretty much all the way to Newport. The 'Audax' route to Chepstow then would be south of Newport past (or over) the recently re-opened Transporter Bridge and up to the Coldra M4 roundabout, then over to the A48 and on the Chepstow. Alternatively get onto the Gwent levels after Newport the Transporter Bridgeand head for Magor/Undy before getting onto the A48.
2. Neath, Cimla, Treorchy, Pontyprydd, Caerphilly, Basseleg, Newport. Slightly more direct but with some substantial cimbs, notably the Bwlch.
3. Swansea to Briton Ferry (via cycle path by the A483). Baglan, Port Talbot then A48 through the Vale of Glamorgan. Not sure how you'd negotiate Cardiff, though. The traditional Audax route from Cardiff is via Rhiwbina, Llanishen, Lisvane, Old St Mellons and onto the A48. This would be a primarily flat option.
I'd be happy to join you for a while, and can give you a few more details of routes, at least at the western end, when you have a better idea of your preferred one.