To my knowledge, what you want doesn't exist. Ocean front property is just so pricey it's mostly used for private residences or big family style hote/motel places (in Montauk).

There's a new place in Long Beach, the Allegria that looks great online, but the reality sounds less than perfect in reviews.
http://www.allegriahotel.com/
http://www.tripadvisor.com/Hotel_Review-g48078-d1237124-Reviews-Allegria_Hotel_and_Spa-Long_Beach_New_York.html

There are quite a few hotels/motels on the ocean or across from it in Montauk that would have views: Guerney's Inn (generally considered overpriced and in deperate need of renovation). The Panoramic View has been renovated and might be nice. http://www.distinctiveventures.com/panoramic-view-resort

The nicest, most romantic places in the Hamptons are unfortunately not on the ocean. A Butler's Manor in Southampton; Baker House, 1770 House, Millhouse Inn & Easthampton Point, all in East Hampton are all lovely but in the village.

This might fit the bill:

http://www.victorianonthebay.com/index.html

Haven't stayed here, but ate a restaurant nearby and saw it from the outside, it looked really lovely.

Agree with Mclaurie - ocean front property on LI is so expensive that only the unimaginably wealthy can afford it. There'e no way a B&B could make a go at rates anyone could pay.

There are condo colonies right on the beach in Amagansett - but they are quite basic and not in the least romantic.

There are lots of hotels on the beach on Long Island. Some are only open in summer. Gurney's in Montauk has been around forever and has great off season rates - sign up for their e-mail specials. There are hotels every mile or so on both the south and north shores when you get about 20 miles from the end of the island. There are also some nice places in Port Jefferson if you want to stay in that cute town.

If you want to stay on Fire Island most communities have small hotels and weekly rentals.

Try tripadvisor for some recommendations. I live here - rarely stya in hotels, but I pass them on my way to where I'm going all the time.

We have stayed at Victorian on the Bay (link above). Not ocean front, but quite near the bay. Even in winter it was quite lovely. I think it would be fantastic in season. The owner is very nice and they serve a wonderful breakfast. Convenient location.

Thanks. Ok, how about either the CT or RI shore?