Links to two hotels I recommended above. El Albergue has convenient meal options including their box lunch and coffee bar at the station. Rates include breakfast so plan train timings accordingly.
NB I prefer to stay in locally owned and managed establishments rather than corporate chains, all I need is a 3-star.

YMMV
Ollantaytambo
https://en.elalbergue.com/food-beverages/

Lima If only passing through then stay at the airport. If you have at least a day to explore, then I like 3 B Barranco especially for a first-time visitor. Note that many flights arrive or depart late at night so often you can leave luggage for the day and head back to the airport about 3-ish hours before your flight (depending on traffic).

(Barranco neighborhood) 3 B Barranco (booked direct ususally). Note that there is no air conditioning which is not usually a problem (January-Feb can be warm) but Lima is typically cool at night. Any noise issues can be mitigated by closing the ventilation window in the bathroom at night (opens to an air well).
For the superb staff, service and my favorite barrio. Especially for a first-time visitor to Lima.
https://3bhostal.com/

I don't have a strong favorite in Cusco, have only stayed in one of the balcony rooms at El Balcon Cusco but I don't think it's suitable for everyone. It's up a steep street (although taxis can reach it) and having some Spanish was helpful. Cusco can be quite cold overnight so you may want to look for lodging with heating rather than heavy blankets!!

Another vote for 3 B Barranco in Lima! But I, too, prefer locally owned and managed hotels to corporate chains.

This was an issue with both our AirBnB and the restaurant we went to the first night. Most of the lodging is not in the old part of the town. So I don't have a recommendation, but if you are are day tripping to MP then there are a couple of hotels near the train station, see another recent post here.

The old town is the real deal, initially wonderful and so "authentic" but the downsides become apparent later: dogs on the loose barking all night which echos in the narrow stone-sided streets, streetlights immediately outside your bedroom window, lots of open burning, roaring motorcycles in the narrow stone-sided streets, perhaps more cultural immersion that you what.

Barking dogs (and loud music) are a phenomenon throughout South America, unfortunately. The worst barking I ever experienced was in Buenos Aires on my first trip. There are things you can do to mitigate, I now try to avoid any street-facing rooms if given a choice, or one can stay in a modern high rise on a high floor. Or a lodging within 'grounds' may limit the encroachment, but no guarantees. Bring ear plugs if noise sensitive.

Tom_mn.
Mlgb.
We are staying out towards the planetarium, more on the edge of the city. Have used earplugs in the past to deal with city noise, which we would prefer to staying in a ‘modern’ hotel. thank you for your input

The name "Aguas Calientes" is a concoction of the tourist industry. The town's real name is "Machu Picchu," sometimes written as "Machupicchu." It always has been. Someone must have decided that was too confusing, so to distinguish it from the ruins themselves, the town unofficially bears the name of the nearby thermal springs.

There is no planetarium in Ollantaytambo, that was my comment where the sewage smell is a problem, the old town there. Earplugs cannot block the noise of dogs barking in the old part of Ollantaytambo because of the stone walls of the narrow streets. It's like they are barking in the bedroom with you.

Not a problem at El Albergue but you are at the train platform so works best for morning departures.

Tom_mn and mlgb. Geez, my error I gave you location on our accommodation in Cusco!! Ollantaytambo we are staying at Nuna Sumaq, which does look to be on the north end of the town. Sorry for the error! 🤪

FYI: Nuna Sumaq Is just feet away from our AirBnB with the nocturnal dogs, very bright streetlights, and sewage smell. This is the old town part of Ollantaytambo, the part with the street grid. This is a 30 minute walk to the train, taxi not easy. There is no vehicular access to this area, must drag bags 300 m over very bumpy cobbles.

Edit: above seems a little harsh, your hotel looks better kept and the problem dogs are half a block away, and there might be someone at the desk all night to chase dogs away. Transport comments stand, access not straightforward. It is a privilege to stay in this old Inca Village, just can be somewhat hard, like when the wind blows and little black flecks fall into the bed (decomposing tar paper from the roofing falling between the sheathing timbers).

The restaurant that also had the ground floor toilet smell (leading me to perhaps see a pattern) is Inka Nato, recommended (except the toilet), cash only. Note that you should bring plenty of soles or US dollars to Ollantaytambo. There are cash machines in Machu Picchu town.