One of Bayamo's most peaceful spots is the Plaza de Himno, northwest of Parque Céspedes. The plaza is dominated by the Iglesia de San Salvador. First built in 1613 and rebuilt several times starting in 1740, the church is famous as the first place "La Bayamesa" was sung in 1868. Its stone-and-wood interior has been restored, and it's open to visitors late in the afternoon, before the 5 pm mass.
The plaza is also the home of Casa de la Nacionalidad Cubana, the town's archives. It's not officially open to the public, but you can ask questions of the staff and maybe have a peek at the antique furniture and interior courtyard. A list of cultural events happening around town is usually posted here.
Southeast of the plaza is the Iglesia de San Juan Evangelista, a church that was partially destroyed in the 1869 fire but whose tower remains intact. The Retablo de los Héroes is a monument to Cuban independence fighters, from Céspedes to Celia Sánchez (who, in addition to being a revolutionary, was also Castro's lover and confidante).