Is there any way to reserve in advance tickets to the Sistine Chapel and the Vatican Museums?

Only by reserving for a guided tour given by the Vatican Museums themselves:

http://mv.vatican.va/3_EN/pages/z-In...zi_Visite.html

Eloise is quite right. You cannot book a simple ticket of admission to the Vatican Museum in advance.

I found the Vatican-sponsored tour to be very unsatisfactory as our guide spoke very poor English in a very low voice.
With a reserved tour ticket, I'm wondering if once inside, one can simply go through on one's own with the audio guide.

I do not think that anyone would chain you to the tour for the duration if you entered for a Vatican Museums-sponsored tour and decided to tour on your own with an audio-guide.

Our guide for the (Vatican-sponsored) Scavi tour (a totally different thing, of course) was very well informed, spoke excellent English and offered a very interesting tour. Looking at her -- young, flowing blond tresses, black pant suit, 3-inch stiletto heels -- would hardly have led one to expect the professionalism with which she conducted the tour.

If the ticket were under scrutiny, I wasn't sure if a 'tour ticket' would pass muster for the admission police should someone want to enter on their own.
I recall paying for the tour ticket at one of the front windows in the main vestibule (I had previously reserved it), before going into the museum complex.

I would take it for granted that, if one reserved a guided tour, one would enter with the guided tour before making one's own way. The aspect with which I am not familiar is the location of the audio-guide rentals with respect to the point of entry of the guided tour. But let's give Chrissem the benefit of a certain amount of savvy on how to deal with that difficulty if and when it arises. Or perhaps s/he does not require an audio-guide and can make his/her way with a good guidebook.

Went early May just got back...... easy entry on the right times but go there first and take the tour exit only exit out of the Sistine and it puts you in place for St Peters. Do not do this exit if using audiophones as it takes a long walk to return them.