the bulk of the work goes to lifting your weight plus the bike's weight up the hill
Yes but uphill you are acting against an accelerative force. So you are having to accelerate your mass which requires more force I would have thought. It's going to be even greater for rotating mass.
I had a heavy steel framed bike that did seem to go downhill better than my aluminium bike. It can't have been because it was heavier though...
Work done (J) is (more or less) Height gain m) * Mass Kg) * 9.82 plus something to allow for air and bearing drag and tyre losses etc
Time take (s) = work / power (W)
When you're climbing a steep hill, usually speed falls to the point that the dominant term is the first. Hence, the time impact of a small percentage weight change is actually less than the change. Rotating mass is a reed herring in this one.
Mike