What would be your suggestion for staying in Baltimore in May? Downtown or Fells Point?

It depends on what you want to do when you are there.

I just stayed at the Hyatt Regency on the Inner Harbor. It was clean and the staff was wonderful. The restaurant had limited hours and the vending machines only sold drinks. Come prepared if you might need snacks. I found the Hyatt to be in a good location. I walked everywhere including over to Fells Point.

You did not indicate when you are going. I was there the last week of October and not everything is open on the Inner Harbor. And other things had limited hours.

I will only be there one night and that is to attend a program downtown in May.

If you have any interest in Orioles baseball, they will be playing home games at Camden Yards against the Boston Red Sox (5/1) and Minnesota Twins, KC Royals, NY Yankees and the Tampa Bay Rays during May.
Depending on how you get to Baltimore, you might want to ride on the Charm City Circulator (free bus) and or the free water taxi connecting 6 docks on the harbor.
Camden Yards is an easy walk from the Inner Harbor.