As far as which building, where would you be more comfortable - in the center of the activity with all the amenities an elevator ride away? I've been in rooms in both buildings, no big difference except the walking distance and the view.
Breakfast - are you a breakfast person or not? If you'd go order a full breakfast every morning anyway, get the breakfast. If you start your day with coffee (in your room) and a pastry (purchased at the excellent grocery next door) and you'd rather save the money, then do it that way. I've been to the breakfast buffet several times - it's yummy, but I personally don't need that big a breakfast every day.
I agree with your choice of location if town is your priority. I don't know how frequently the hotel ferry runs, but if you're using it to visit the beach club, it's going to involve the ferry across the harbor, a trolley shuttle ride up over the hill to the beach club and then back - public transport from town to the beach club might wind up being a lot more convenient. It's not expensive either.
As far as the 'out of the way' experience -give me a little more of an idea what you're looking for - shopping? nightlife?hikes? parks? beaches?

Oh thank you! We did figure that we would rather be in town and probably only go to the beach for a while each day. Any not over the top restaurants - my husband will not mind wearing a jacket but the tie will put him over the edge - I guess what i mean is we know about the caves but any other natural things that are worth the trip? Is the shopping at Dockside worth it? Is that a place where might find a good restaurant. We like to have wine with dinner but do not like to eat like 5 courses but would like to go somewhere nice!! It seems like we can take the bus/trolley to the caves and anywhere else we micht want to go. Thanks so much for the info on the hotel.