Take a look at Grenada.

I think you would be happy on Virgin Gorda. We've been 3 times, first time when kids were little. It's small enough to give you that real island feel . . . you get to know the locals at the grocery store and at the gas station and they wave at you when you drive by. There are miles of nearly deserted beaches to explore yet also several upscale resorts (Biras Creek, Little Dix Bay, Bitter End) where your husband could book a massage etc. Some great small bars where expats, locals and tourists mingle (Hog Heaven, Mad Dogs, Fat Virgin) You could rent a small villa or condo unit at Nail Bay and be center island (I find renting a home like place was easier when the kids were little, nice having own kitchen and separate bedrooms so they could go to bed early while we sipped rum drinks on the porch!) Message me if you want access to my photo diaries and trip reports.

Great suggestions, thanks. we were actually looking into virgin gorda.

To Tamsentm - I'm very interested in Virgin Gorda as well. We would want to stay at a mid-size resort. Not all-inclusive, but something with some property and space. Any suggestions? Thanks!

I'm pretty familiar with VG. There are nice (read expensive) hotels like Little Dix, Biras Creek and Bitter End, but nothing much at the mid-low end of the price spectrum. There aren't any 'all-inclusive' places. I'm not sure if they planned it that way, but few hotels does translate into a very authentic (non-touristy) Caribbean experience with no crowds and unspoiled beaches.

We rented a villa in Nail Bay (Rambutan Villa) off VRBO and split our time between eating out and eating in the villa. Like Tamsentm said, it was easy to put the kids to bed, either for a nap or the night, and still continue vacationing. The spa and restaurant at Little Dix are the same price if you are staying there or dropping in. Biras only offers services to guests and, like the Bitter End, is difficult to get to.

Be warned, VG isn't the cheapest destination in the caribbean but you definitely get what you pay for.

Hog Heaven is an absolute must for anyone going to VG. Unbelievable views and great BBQ.

Our family went to both Tortola and Virgin Gorda. Loved both, but VG was more unspoiled(not that Tortola is...Tort is just a lot bigger!)

We rented a terrific villa at Nail Bay that had a private pool and was a short walk down to the beach, which had great snorkeling at both ends! We ate breakfast and lunch at the villa. Lunch, especially, was nice, as we hung around the pool and indulged in the occasional tropical rum concoction. Your son could nap in the afternoon, while you and your spouse could relax by the pool..

Every evening, we headed out to dinner...never had a bad meal. We did some hiking in the early mornings, went to the Baths, took a catamaran excursion...all in all, a wonderful vacation!

What about Costa Rica?
The locals are so friendly, there is a great combination of active adventures and relaxing beaches (some of which are very secluded) and there are monkeys!!