Cambridge also has Madingley, the American World War II cemetery and one of the things the Americans still do well. The hop on/hop off bus tour is a great way to get to Madingley and also get a feel for the city; the market is fun to poke around in and the city has great book stores.
It's definitely worth paying the fee to visit at least a couple of the colleges.

Do either of then have impressive libraries that one can visit?

They both have impressive libraries, thats why they are considered two of the worlds best universities!!
Oxford boasts England's oldest library at Merton College which you can tour for £1. Harry Potter's Hogwarts library is Duke Humphreys which is also open for private tours along with another Hogwart's schoolroom, the Divinity School.

frankie, grsing, and henneth - thanks for your input!