Hi,
couple of suggestions Colombo to Pinawella elephant orphange is fine, then, and next day to Sigirya rock fortress.
But,, after visiting Sigirya you'll be hard pushed to get to the Tea Factory hotel. Never stayed there but I believe it's near Newara Eliya ?. I'd suggest instead, you visit Sigirya in the morning, lunch on the road somewhere, plenty of places en route and in the afternoon vist the Budhist cave temples at Dambulla. Then into kandy for the night.
Plenty of choice of hotels of all types in Kandy.
If you're still mobile after the day, you can visit the Golden Temple of the tooth in the evening. There's also a cultural show every evening , in a small theatre almost next to the temple. Kandian costume/mask dancers, firewalkers etc.
Personally, I wouldn't recommend Yala. Very expensive to be thrown around in a jeep while it chases loads of other jeeps around the tracks with the drivers trying to find an elephant to show you. Too many jeeps, few animals and high cost.
Instead of yala Id' suggest you visit Ella. The road down through the hill country, ( the same one you'd take for Yala), goes through Ella . Amazing scenery all around and fantastic walks through tiny lanes , hills and tea plantations. Try the walk to Little Adams Peak or Ella rock. 1st is easy, 2nd is strenuous.
From there you can get back the Light house Hotel on the road through Ratnapura, the gem digging centre of Lanka.
If you want the info on a couple of guys who are self employed Driver?guides email me ( RBoakes@aol.com) and I'll send it along. I've known both for a few years and know they are both excellent , reliable and far better than a big company employee. Both will arrange everything for you by email.