The best beaches and safest beaches for swimming are not vendor free. Medano Beach in CSL is the best for swimming and water sports but has vendors. Along the Pacific side are several resorts such as Capella Pedregal and Pueblo Bonito. These are beautiful resorts on isolated beaches but not safe for swimming. We stay at PB Sunset Beach and love it.

La Paz hands down. The most Mexican of the 3. Los Arcos just South of town has a great beach, pool, a water sport center where you can Scuba, snorkel & sail. Visit manta Rays, whales ect,
A hotel but check out the Condos there. We rented a wonderful 3 bedroom 3 bath, dining room, kitchen, den & living room with a large porch for around a $100 a night.
La Paz has a excellent Malecon lined with good restaurants, a very decent shopping area, Plaza & a interesting Museum.
One of my favorite places.

PS VERY VERY safe swimming.

I agree! La Paz fits best what you ask for, but you need a car to get to the really nice beaches. They are to the north and south not actually in town. I loved this hotel: Club El Moro, it's not a "resort" just a cool hotel at the edge of town
http://www.clubelmoro.com.mx/

To clarify your title... Cabo San Lucas is a town that is part of "Los Cabos" which consists of San Jose del Cabo, the tourist corridor strip of resorts in between, and the town of Cabo itself.

Ocean swimming is very rough and dangerous in most places Medano Beach right in Cabo is one of the few places it's safe to go in the sea. Yes there are vendors there, and cute beach bars & restaurants.

Suze We had no car but hired a cabbie with a Van for our Party of 5 to tour the East Cape, Cabo & elsewere.
He took us to dinner at some very good out of the way places in La Paz proper. Do not recall his name or the cost but it was very reasonable. Far cheaper than renting a car.

Stewbear, but I like to go to the beach every day. And the beach right in town isn't suitable for swimming or sunbathing really, which is my only negative comment about La Paz.

I was there for two weeks solo and without a car, and if I returned to La Paz would definitely want one. I don't think car rental would be more expensive than hiring a taxi or driver every day, do you?

No you are correct Suze. There really is no beach in town that's why when we brought our wifes ( my daughter) we stayed at the Condos at Las Arcos but 5 minutes away. It also is on the local bus line.
When I went with my dive buddies we stayed right in town as after a day on The Gulf we enjoyed the Malacon.
The cab to Laos Arcos was about 3 dollars US as I recall, but it's been a few years.
The big thing to me is the OP is looking for the least "touristy" location of which believe you agree is La Paz.
The main reason I have not been to Baja for a few is frankly the weather is limited. December was nice but February Freezing! From friends I am told summers are stiflingl.
I now only do Mountain Mexico in the Summer. Of Course PV & South in the Winter for "beachen".