India: Planning for this summer

(which, I know, is the worst time to go...but it's what I have.) This could either be from June 09-on or July 03-on for three weeks; I'd love feedback on which would be a better choice in your estimation.

Anyway, my plans look a bit like this at the moment:

Fly EWR to Delhi; stay three nights at either Master Paying Guesthouse (if I can get it) or Delhi B&B (likewise)

Fly to Varanasi, three days there; overnight train to Agra for one night to see the Taj Mahal and Fatehpur Sikri. Train back to Delhi, fly to Trivandrum for five days or so in Tamil Nadu. (This part of the trip would be visiting, and I would be escorted by the people I'm visiting. This is also the part that would be cut if I were to need to cut it back to 16 days.)

Fly back to Delhi, finish the trip with a week in Ladakh, resting for a couple of days and doing some light trekking/monastery visiting/Indus River rafting maybe. I've had recommendations for a few good guesthouses here, but I'd love to hear even more.

Anyway, it sounds like a bit of backtracking; I do want Ladakh to be at the end, though, but any other rearrangements would be fine. Any info would help!

(By the way, I'm 42, have traveled solo in China/Tibet as well as a number of other places, and am reasonably fit, as in able to walk for hours with no problem, not so good at scaling rock.)