Smaller towns are nice but with just 5 days and never been to Italy I'd spend those in Venice or Rome or both.

You are planning probably your first trip to Europe and doing a quick hit of hotspots - if you cut down the number of countries you would have time for lesser beaten track places but that will be for next time.

"...to celebrate my boyfriend and I's 3 year anniversary"

I have to do it. It's MY, not "I's".

"My boyfriend and my 3 year anniversary sounds wierd to me...too many 'my's just did not sound right to me. Anyways thanks... haha

I think it's unlikely you're going to do a 23-day trip for $4K, including airfare.

Let's subtract $1000 for airfare, leaving you with $3000. Divide that by 23 days = $130/day for your share of accommodations, food, travel between locations and costs/admission for attractions. And note that some of your time will be spent at some of the most expensive cities in Europe at the most expensive time.

That said, I would suggest:

-- Cutting down on the number of destinations, to cut travel costs.
-- Renting apartments, rather than staying in hotels. You can find some basic one-bedroom or studio apartments for much cheaper than hotels, plus you'll be able to prepare some meals and save money that way.
-- Do lots of homework on how to save money on attractions and city-to-city travel. (You've already started on that.)

Stay in hostels and not hotels on that budget.