Another vote for go for it. When I first decided to do something about my weight (weighing in over 21st at the time (don't know how much more, that was the top of the scales
) I started with walking, but very quickly decided to try swimming.
I found the walk from the changing area to the pool terrifying, but once in the water it felt more comfortable as I could sort of hide my body under the surface. Soon I started to get more comfortable, especially as there were other fat people there swimming as well and it got easier to deal with, meaning I started to enjoy myself more. It's a shame there isn't really decent swimming provision round me at the moment.
Also individual weighings can be massively affected by salt intake, or even eating a meal later in the day so it isn't through the body by the time you go to weigh yourself. I've found it is a lot more comforting to record daily weights in a spreadsheet, and then use a 7 or 10 day rolling average weight to try and iron out the noise and show whether the weight is sensibly going up or down. (I had a range of about 2kg in my weighings last week)
I really advise reading this website, more specifically the 'signal & noise' and 'weight monitoring' sections: -
http://www.fourmilab.ch/hackdiet/e4/It's not a diet as such (though it does do some calorie counting if you read some of the later chapters - I haven't really), more an process monitoring approach to tracking weight loss.