One of the things that can be most intinidating about cycling in London, can be finding a cycle-friendly route. Two sites that I've found very useful are:
TFL Journey Planner
London Cyclemaps (down at the time of writing)
My impression is that the Transport For London site (you have to go into "More Options" and select "Cycle") seems to favour official cycle routes (either on roads, or through parks, towpaths, etc), whereas Cyclemaps prefers backroads. But the best thing to do is try both, and pick the one (or combination of both) you prefer.
And if you prefer paper maps, don't forget you can get free cycle maps from LCC and TFL (and often your local bike shop)