Traveling to these cities via bus is the cheapest option, but it's only for the truly adventurous, because local schedules are difficult to come by and service is spotty. If you're taking off from Delhi, the bus stations are large and can be chaotic and hard to deal with. You can book a seat on a private, air-conditioned bus for travel between the cities through any local travel agent.
Within cities, local bus lines are the only form of public transportation. In general, they are too crowded and difficult to use for the average foreign visitor.