I think you'll be going against the commuter traffic in both directions. I would leave Milan as early as you can get yourselves going in the morning. The few hours you have to spend on the lake will go by quickly.