What you will ultimately recall about Cape Town is the sheer grandeur of its setting—Table Mountain rising above the city, the sweep of the bay, and mountains cascading into the sea. You will likely spend more time marveling at the views than anything else. Francis Drake wasn't exaggerating when he said this was "the fairest Cape we saw in the whole circumference of the earth," and he would have little cause to change his opinion today. You could spend a week exploring just the city and peninsula—and a lifetime discovering the nearby wonders of the Western Cape, including the Winelands, one of the great highlights of a trip to South Africa.
Cape Town has grown as a city in a way that few others in the world have. Take a good look at the street names. Strand and Waterkant streets (meaning "beach" and "waterside," respectively) are now far from the sea. However, when they were named they were right on the beach. An enormous program of dumping rubble into the ocean extended the city by a good few square miles (this can, no doubt, be attributed to the Dutch obsession with reclaiming land from the sea). Almost all the city on the seaward side of Strand and Waterkant is part of the reclaimed area of the city known as the Foreshore. If you look at old paintings of the city, you will see that originally waves lapped at the very walls of the castle, now more than half a mile from the ocean.
