We have a 3-day stopover in Tokyo in early September.This is our first time in Japan and we are wondering whether it is worth taking a shinkansen to Kyoto for the day?

Have you checked the shin train times on hyperdia.com to get an idea of the earliest they run to Kyoto and the latest they run back to Tokyo? It would be a very long day, but depending on how early/late they run it can probably be done. The only way I would do it is if I had one area in mind (like Hagishiyama, the couple of very nice temples there and the lovely walking streets and the shopping would be a great day, give you a flavor of the old, and in a nicely confined area), or a tour set up that would take me right to places. Kyoto sites are spread out (w/ exception of a few areas like Harishiyama where you can walk btwn temples, and the path of philosophy area), and if you do not limit yourself to a confined area w/lots of sights and things to do, you will spend time getting from one to the other, even if it is by cab. And you may have a hard time gauging how long each thing will take or how far apart they are, etc., at least I did at first when I got there.

geez, it is Higashiyama. long day, too much typing.

It will cost you 6 hours and about $250. You decide.

Oops, good point. I forgot about the cost since you are only in the country 3 days (we had JR passes so cost was not was an issue)
It certainly isn't optimal. An alternative to consider might be Kamakura as day trip from Tokyo, for temples, etc. I was going to go there but decided we'd get that in Kyoto. It isn't Kyoto but it has some flavor.

I'd save Kyoto for another time when you can do it properly. If you want to take a day trip from Tokyo, Kamakura, as already suggested, is good. Other popular destinations are the Hakone circuit, and Nikko. These are further than Kamakura, and Hakone is a more complex journey (Shinkansen, then local train, funicular, gondola, boat and, finally, bus back to the Shinkansen).