Best I can suggest is to head to Eynsham on the back road, over the toll bridge (free for bikes - or used to be) and come in to Oxford on the Botley side. Bus/cycle lane from there to the railway station.
Witney > South Leigh, pick up B4449 towards Eynsham, follow this to B4044, I'd then fork right towards Cumnor on the 4017 and the from Cumnor, down Cumnor Hill - wheeeee - into Oxford but you can just carry on on the 4044 under the A420 and come into Oxford close to Seacourt (Decathlon!).
Two reasons for the detour via Cumnor, makes the trip a little bit longer and the freewheel down Cumnor Hill.
I believe there _used_ to be a ferry at Babcock Hythe, but that it ceased operation decades ago. Haven't been that way myself so can't be certain about that.