I topped an old thread which has some great ideas for activities in Bermuda. It is a small island, but you can find some private spots on the beach if you will continue on towards Hamilton after passing Horseshoe Bay, which is far and away the most popular beach. They will be on your right and you can just swing down to them on your moped, park, and walk a bit away from the parking lot and you will find your private paradise. The ocean in May is a little chilly for me, but you may luck out.
You can find a tea at the Fairmonts. On my last visit the Dining Room at the lighthouse was serving tea, but that restaurant has recently changed hands so I don't know if there is still a tea offered.
Casual lunch? Try HogPenny Pub, the Frog and Onion, Angelo's Bistro, Lemon Tree or the lounge at the Southampton Fairmont (great view, excellent pizza and smoothies).
Do take a walk along the Bermuda Railway Trail, it is a lovely way to see some of that gorgeous island.
Have a wonderful trip!

Just returned, we enjoyed the Aquarium and Zoo in Flatts Village between Hamilton and St George especially the lemurs in the Madagascar exhibit and the seal feeding. Best lunch (rockfish) and draft beer(Dockyards IPA) we had was at Pickled Onion on Front Street in Hamilton.

If you have scooters you will have a chance to get away from the pure tourist spots and get a look at island life. Everyone we met was friendly and helpful. Be sure to say "Hello" or "Good Morning, Afternoon, or Evening" and you will get a nice response.

It’s on the opposite side of the island but I’d personally take the bus over to St. George’s one of the days.

I’d head over to Tobacco Bay and St. Catherine’s Beach. Blackbeard’s at St. Catherine’s beach is good for a nice casual lunch with some breath taking views. There are a number of moderately priced restaurants on the water downtown as well….The White Horse is a good option.

You can also walk around town, do some sight seeing (lots of photo ops) and do some souvenir shopping.

Does this make sense..
Wednesday - Ferry to St George , spend some time there during the day, head over to Tobacco Bay, then Hamilton in the evening for Harbor Night
Thursday - scooter rental. Explore Sandys parish.. Somerset, the worlds's smallest drawbridge, maybe some hiking on the railway trail - looking for suggestions as to the best part, and where we can leave our scooters. Is there somewhere we could leave the scooters, hike and then take a bus back to where we left the scooters? Possibly also ride down to Gibbs lighthouse...
Friday - since we leave on Friday spend the day in the Kings Wharf area, including lunch at the Frog and Onion
Still looking for somewhere less upscale than the Fairmont for afternoon tea, that would fit in to the rough itinerary I have laid out above. Unless, of course, you experts can suggest a better itinerary!

another tea option
http://www.tripadvisor.com/ShowTopic-g147255-i208-k5390844-New_Option_for_Afternoon_Tea_in_Bermuda-Bermuda.html

You might consider leaving your bikes (properly locked!!!) near the Somerset bridge and walking the trail. You should be able to catch the bus on Middle Road when you are ready to return.

Your itinerary looks fine to me, but I suggest you be very flexible and take advantage of beautiful sunny days!