Last October, I tried a budget-friendly chain called "Tendon Tanya" which had been mentioned by someone on this board.
Tendon Tanya has many locations in Tokyo TOKYO LOCATIONS | TENDON TENYA,
and menu http://www.tenya.co.jp/pdf/global_menu.pdf

The actual food pieces were smaller than what were depicted in the pictures, so ordering a large set meal turned out to be just right. The place I went to was near UENO station. The food was good, the eatery smelled a bit of frying oil, so I was happy to quickly finish and go.

We sought out Tempura Tsunahachi Honten, in Shinjuku on our first trip and weren't disappointed. It's part of a small chain, but I believe this is the original / main branch (honten).

Address is: 3-31-8 Shinjuku, Shinjuku-ku, Tokyo 160-0022

I think it's about 20 minutes from Chiyoda on the Shinjuku line.

FWIW -- I found tempura widely available throughout Japan, and every single bite I tried was delicious. Maybe that simply says that I'm not a true aficionado.

We ate tempura at lots of places, and with the exception of just one tiny grandma and grandpop place, the tempura was always great. In that case, the oil temp wasn't right and everything came out sodden with fat and a bit soggy.
I'd say the place I've mentioned above for me, was one of the best we've tried thus far.

I've had quite a lot of 'claggy' tempura in Japan. Tsunahachi in Shinjuku recommended by Kavey above is an old favourite, always good and offers a vegetarian menu which I believe is unnusual. Tenya is one of my alltime favourite spots for a quick meal - but be under no illusions, this is cheap fast food. My favourite approach to tempura is to sit at the counter and order piece by piece rather than the set menu. This works out cheaper in my experience and I get exactly what I want , but then I go heavy on the vegetables. Tenichi (honten in the Ginza, many branches) is always excellent. If you look in the Michelin guide, there are one star tempura restaurants - this means not break the bank level but special. My favourite of these is Hasegawa, very convenient if visiting the Hokusai or Edo-Tokyo Museum. Remember, lunch is always a good deal. Any of these are within a 30 minute subway ride from Chiyoda-ku (which is a pretty big area, mind!)

Thank you to everyone for all the wonderful suggestions!

Here is a trio recommended by my hotel concierge; I've not booked at any of them because I am doing my own research. The hotel is near Tokyo Station, to give you an idea of location:

Tenichi 天一
Known as Ex-President of Unites States Bill, Clinton visited and his favorite one.
Within 10 to 15 minutes by taxi from Shangri-La Hotel, Tokyohttp://www.tenichi.co.jp/mainshop/

Tenkuni 天國
Enjoy thinly battered and lightly fried original Tempura.
than 100 years. Tenkuni has been loved by local people as well as many VIPs from all over the world.
Within 10 to 15 minutes by taxi from Shangri-La Hotel, Tokyohttp://www.tenkuni.com/english.html

Tempura Onodera 天ぷら小野寺 http://matsubara-an.com/index.php#shoplist
Vegetables are fried at a lower temperature to maintain their
freshness and colour, while seafood is fried to a crisp
Within 10 to 15 minutes by taxi from Shangri-La Hotel, Tokyo

https://onodera-group.com/en/tempura/



The ones that I have turned up so far, with my own contacts, and with a request to be within a walk of my hotel, are these; all are well reported on Tabelog; they may be pretty expensive, though:

Sonoji
Kando
Fukamachi