I think for a "cheap, cultural" breakfast, I agree that you may have to leave your hotel neighborhood. Take the subway to Union Square/14th St, and go to the Grey Dog cafe on University Pl, or any one of the other 'local' coffee places in the village/chelsea/grammercy areas.

We ate at Penelope a few years ago after waiting in line for 15 minutes for a table on a Sunday morning. I don't 'get' its allure either. Overpriced and mediocre was the general consensus at the time but it was packed with customers. I never went back but have wondered how it's doing.