I recommend The Tea Set at 235 West 12th Street between Greenwich Street and 4th Avenue. A variety of teas, desserts, and the food is good, filling, and mostly organic. Friendly owner. Nice place.

Agree with YK -- you will find everything very nicely priced vis-a-vis your CHF!

Not big on tea and cakes.

But for wonderful inexpensive Italian pastries go to Veniero's for table service.

At Chelsea Market, Fat Witch makes exceptional brownies and Amy's makes delicious breads and cakes.

In Manhattan, "budget" and "afternoon tea" shouldn't be in the same paragraph, much less the same sentence!