I would recommend the metro toronto zoo

Will you have a car? If not, skip the zoo, much too long to hike up there on the bus with 2 little ones. Check out The Beaches. Take the Queen street streetcar out to the beaches (make sure they go al the way to Woodbine, the start of the beaches before getting on, 1 goes all the way, one stops a few blocks short). The Beaches is very family oriented with lots of cafes, parks, a kid oriented library, family restuarants - hubby can meet you for dinner out there. Walk along the boardwalk right at the beach.

The Ontario Science Centre at Eglinton & Don Mills - the kids will love it.

Harbourfront is also a great place for kids - probably a wading pool there.

Also, take the ferry over to Centre Island - Centreville is there with tons of rides for little kids. Plus it is a great park to walk around, have a picnic. you can get the ferry at the very bottom of Bay Street (@ the Queen's Quay). The ferry to Wards Island will take you to the village of artistic people living on the island - neat houses, but Centere Island is better for the kids.

Hope this helps.

Skip the zoo. It's too big for little kids, most of the animals are too far away, and and it's too far to get to, anyway.
The kids favorite will be The Children's Own Museum, immediately south ofthe Royal Ontario Museum, on Queen's Park, just south of Bloor Street (Queen's park changes its name to Avenue Road when it crosses, and is north of Bloor) You want the Museum subway stop.
After you visit the Children's museum, go up to Bloor Street, turn east (right) and go to Chapter's Book store, and take the boy's to the kid's department on the third floor. After that, go out the back door of Chaptr's and let them play on the huge rock out there.
The suggestion of Centerville on Toronto Island is excellent; the boys will love the ferry ride, too.
Harbourfront has lots to look at, and this is probably worth a second trip, since the kids will be tired after visiting Toronto Island.
The 4 year old will like the Ontario Science Centre, but the two year old won't get much out of it. On the other hand, it's as good a place as most if he just wants to run around.
If you walk west from the hotel along Queen Street a couple of blocks to John Street, they'll enjoy seeing a truck driving out of the wall of the CityTV building on Queen Street West (look up as you pass the CityTV parking lot) and then turn north on John (there's a Starbucks on the corner) and walk a block to the park. There are slides and swings and other little kids things there. That's the closest park to the hotel.
Checking with my five year old, he reports he likes to ride in a boat in the harbour (the ferry to the island takes care of this) and he likes to ride on buses and the subway.
Toronto has open-top double decker tour buses, and your kids will enjoy riding up there and just looking around.
I had lunch in a new steak house in the hotel today; Le Bifteque, and I would bet they've got alcoves or sections that would be in the restaurant but fairly private. We were impressed with our food; good solid real steak and potatoes and deep fried onions, plus lots of other things on the menu.
If you want more trendy food, there's a new restaurant, very elegant, looks very expensive, called Tundra, in the Hilton, which is about a block away.
Hy's, in the Richmond - Adelaide Centre, (again, a block away)is another fine steak house that's used to hosting groups of a dozen. It will cost 20 -30% or so more than Le Bifteque.
BAK

Did anyone mention Ontario Place yet? - some of my best childhood memories are from there.
Make sure you take them for a ride on the subway and/or a streetcar! perhaps up to a park in the neighbourhoods, or High Park - they play and you read a book and sit in the sun!

The High Park suggestion is a good one. In addition to the regular playground near the Bloor Street (north side) main entrance (which is just across the street from the High PArk subway station) there's an adventure playground in the southeast (lower left when you have your back to the subway) corner of the park, along with a crummy little zoo the kids will like anyway. And the food's pretty good at the little takeout restaurant next to the playground.
BAK