I don't know about a boat ride in December. If you really want, maybe, sometimes they don't run if it is too cold and there is ice on the river, I think.

It is pretty convenient to take the RER B down into the city and just get out at the St Michel/Notre Dame stop. You are right there for Notre Dame at the Seine, and there are a couple boat cruise companies that have docks right near there, also. There are enough cafes right along the Seine there which you can see (I'd walk a bit east of the RER stop to get a little more open space, there are a couple set back. Or you could walk over to Ile St Louis and try to find something on it, that might be a good idea. That is the Latin Qtr by address (where you get off the RER, not the island), but I'm not sure what you are thinking of by "going by" it. It is an entire neighborhood area, some parts more interesting than others, but it isn't just something you can go by.

Here's a brasserie where you could eat on the island and have a good view of Notre Dame http://www.labrasserie-isl.fr/

Here's a good small restaurant in the Latin Qtr just a bit off the quais in that general area: http://www.restaurant-navigator.com/

You'll see the top of the Eiffel Tower from most places, and see more if you do take a Seine cruise. Given the time of year and your time limitations, and that you will have a baby, I personally wouldn't bother going all the way over to it from the Notre Dame area. For what? Just to look at it? If you want to spend your time in line to go up it (and if it is cloudy and grey, no views will be great, you would have to go over there to do that. I wouldn't.

Christine outlined some good ideas for you. Once at Notre Dame I would wander to the Ile St. Louis for atmosphere and lunch. We like Au Lys d'Argent which has crepes, soup and great chocolate cake.

You'll be near lots of charming streets on the island and near St. Michel for walking, cafes to duck into for a coffee or chocolate if it gets cold and you need a rest. If it's not too cold you can walk to the Louvre for a photo opp.

If you time your Seine cruise right you'll see the Eiffel Tower's sparkling lights as well as float past the Louvre and Musee d'Orsay. I have taken this cruise in December and even braved the outdoor seats but there are inside seats for the baby. My cousin visited in December some years ago and the Seine was frozen over so the weather can be a factor.

Hope you have a lovely afternoon and evening in Paris.

Thank you very much for the tips!! That truly helps!! : )

if time permits, you could also get the "batobus" up the Seine from Notre Dame to the Eiffel Tower, but if you want to go up it, you'd need to pre-book.

if your infant is toddling, there is plenty of opportunity for him/her to do so in the Tuilleries next to the Pyramide entrance to the Louvre.

The park behind Notre Dame is a good place for a toddler to stretch their legs.

On a short day, not sure that squeezing the Louvre in will really work. Consider beautiful Ste Chapelle instead...very close to Notre Dame and a favorite of mine. I would look for a cafe along the rue St Louis en Ile for lunch. Isle St Louis is very charming. I would prefer not to cram in too much, but see a few things closer together so that you are frazzled.You could also take in the deportation Memorial which is also behind Notre Dame.

You can take the bridge over to the Latin Qtr to check that out.

If you really want to see a museum, consider L'Orangerie. Amazing Monet canvases without the crowds.

If you can do a Seine boat tour, then see the ET that way.

Thank you very much!!! Love the suggestions, I will get a map and make a list now: ) this is so helpful!

If you are near the Eiffel tower you could walk back down Champ de Mars to the area around Ecole Militaire subway stop, there are numerous eating places around there! I cannot vouch for any place in particular.
The Montparnesse tower is a good option which gives a great view of Paris, if an elevated view of the city is to your liking.Its also enclosed and much better if your day ends up a bit showery or rainy(hopefully not