Seattle-Butchart in a day... while physically do-able, I wouldn't want to do it. Let's say you take the 8:30am boat; you arrive in Victoria at 11:30am; with customs and the time it takes to rent a car/find a tour bus, you'd only be at Butchart's I would say, at best, at 12:30am (the Gardens are maybe half-an-hour north)... Then you spend 2-3 hours, come and catch the 5:30pm or 7:00pm return after possibly a brief stroll in downtown Victoria (remember that people are lining up for the Clipper approximately an hour in advance at both ends). Sounds exhausting and rushed to me.

If you must make it a day trip from either Vancouver or Seattle, Vancouver is your better choice, definitely closer to Butchart's (if the Gardens are you're only interest, you could directly from Swartz Bay which is quite close to the Gardens, rather than going the extra 30 minutes into Victoria), no customs & immigration (plus, if you take a bus, there are buses hourly, which gives you greater flexibility in terms of schedule).

However, like all the others, I would still spend the night in Victoria for a more relaxed visit where you'd not always be looking at your watch...hey, Victoria's a really lovely small city, in my opinion possibly the most beautiful city in all Canada; I spent 3 nights there and wouldn't have minded more!

Daniel