If you like modern art, then take a trip up to Dia Beacon, a wonderful museum in, of course, Beacon.
It's about 75 miles from the city.

HowardR, Dia has been on my to do list for a while.
Any suggestions for lunch and other activities to make a whole day special event out of the trip?

I don't have any recommendations in the Beacon area, but you could drive down to, say, Cold Spring for lunch or dinner. If that appeals to you, I can pass along a suggestion or two in that area.

I purchased bus tickets on Boltbus to NYC from DC for $32 RT for me and hub for Valentine's Day -- that's $16 each RT! (They offer one seat per bus at $1.) The reverse would make a pretty good day-trip to Washington and only $2, if booked ahead of time! (I ended up repurchasing the return for $40 when I found a $99 hotel deal.) Plenty to see and do for free on the Mall, not far from the bus drop-off point.

Well... though I have enjoyed my trips to DC, it's not exactly what I had in mind for a closer to home day. Thanks anyway,SoloNoMo.

Topping for some other suggestions.

Drive or train to Red Bank, NJ. Have a bite at Eurasian Eatery (or one of the other good restaurants in town) and then catch a show at either Count Basie Theater or Two River Theater.

http://www.eurasianeatery.com/home.htm
http://www.countbasietheatre.org/
http://www.trtc.org
http://www.escapemaker.com/nj/redbank/redbank.html

I would recommend a trip to New Haven on MetroNortha tour of the Yale campus.

I was going to suggest Sunnyside and Van Cortlandt Manor but just checked and saw they are closed this time of year

If you go to Yale, their Center for British Art is great. http://ycba.yale.edu/collections/coll_p-s-index.html
Some options from Metro North for day trips including New Haven.
http://as0.mta.info/mnr/getaway_tickets/get_event_outbound.cfm

If you're interested in discount shopping, Woodbury Commons (drive or take a bus from Port Authority) or one of the malls on Long Island. The Hampton Jitney has service to the Tanger Mall in Riverhead but there's now one in Deer Park which is closer.

If you've never been to Teddy Roosevelt's house, Sagamore Hill, that could be nice http://www.nps.gov/sahi/planyourvisit/hours.htm with some shopping at either Roosevelt Field or the Tanger Mall in Deer Park.

Tourists do day trips to Philadelphia by train all the time.

I'm not sure of the distance, but I think the following would be within your proposed range: Hyde Park. Make reservations for lunch at one of the restaurants at the Culinary Institute of America. Tour FDR's home, visit the FDR library and tour the Vanderbilt mansion (some of this before lunch and some of it after).

Hyde Park is nearly 100 miles from the city.
I'd save a trip there for the warmer weather, as you really want to spend time outdoors at FDR's home and museum and the nearby Vanderbilt Mansion.

It's 112 miles, according to Mapquest, and 2 hours, 15 minutes, BUT... Jim Thorpe, Pennsylvania. You mentioned funky and unusual - there are some cute restaurants and different activities in an old coal-mining town. You can tour the Asa Packer mansion and find some funky antique shops, etc.

Totally different from NYC, which is always fun, right?!

Just realized my post about New Haven was cut by mistake. I had meant to suggest the Peabody Museum and maybe a tour of the Yale campus. I have never seen the British Art collection myself but it looks great