The downtown core is a mishmash of modern and old buildings connected by a series of large plazas, four of which were designed to form a cross when viewed from the sky, with the cathedral in the middle. Though some remain, many colonial-era structures were razed before authorities got serious about preserving them. Conservation laws, however, merely prohibit such buildings from being altered or destroyed; there are no provisions on upkeep, as plenty of abandoned, crumbling buildings indicate.
Must-visit sights include the Palacio del Gobierno and the Instituto Cultural Cabañas; both have phenomenal murals by José Clemente Orozco. Even if you're not in the mood to shop, you should experience the bustling Mercado Libertad. Explore the district in the morning if you dislike crowds; otherwise, you'll get a more immediate sense of Mexico's vibrant culture if you wait for street performers and vendors to emerge around the huge Plaza Tapatía in the afternoon.
Allot at least three hours for the Centro, longer if you really want to absorb the main sights and stroll along the pedestrian streets.