You might wish to go from Granada to Cordoba for a day or overnight(before going to Seville)- takes about 2 and a half hours.
Cordoba to Seville , 45 minutes.
If you stay in Toledo until Monday, you still have to return to Madrid to continue your trip.

I was contemplating picking up a rental in Madrid, driving to Toledo and staying overnight, and then continuing on (via car) to Cordoba, then Granada, then Sevilla.

Is this a crazy concept?

I know there is the AVE from Madrid to Cordoba/Sevilla; so ... that was my original thinking. But perhaps a car is an option.

<Is this a crazy concept?> Not at all. We did that in reverse.

But you could go to Granada first from Toledo.

A nice old town to visit just north of Granada is Ubeda.

Then Cordoba and Seville.

driving until you reach Seville is a good way to cover that part of the trip....gives you flexibility .
In Granada, book a hotel with parking !

Thanks all. Will look into that.

Crazily, it's cheaper to book a car for 7 days than 5!

Looks like a great trip well planned, Cathy has great advice too.