What is the cheapest way to book a train to Budapest. I want to make the arrangements this week for our travel in late April. I was told that using rail Europe is more expensive. Please pardon my ignorance as this is my first trip to Europe.

To Budapest from where? Try this link

http://reiseauskunft.bahn.de/bin/query.exe/en

If you want to keep the costs to the absolute LOWEST level then wait, if you can, to book any seat reservations, buyt tickets, etc., AFTER you arrive in Europe. Why? Because agencies like RailEurope, RailSaver, etc., will either tack on shipping and handling fees, mark-ups of seat reservations and/or tickets, etc.

IF you feel you cannot wait please let us know your proposed itinerary, how soon you will be traveling once you arrive in Europe, etc., so we can help you further.

I will be arriving in Prague on April 21 and have to catch the first train (or bus) available to Budapest to arrive either on April 21 or the April 22 at the latest. Is catching a bus an option, what would the cost be?

There's an overnight bus. See http://www.bei.cz