The roads in the region are small but normally fairly well trafficked. Expect waiting time to be anything from 15 seconds to 15 minutes (in rare cases it could be more) and people to be friendly to hitchhikers. Theo thumbed the region for 2 weeks and had a really good time doing so.
The area is full of tourist buses, driving along the many different sites. Don't expect to get any rides from them, rather than that focus on passing locals, who will always be up for a chat.
User:MaxHermens hitchhiked the same route the tour buses take and recommends it. Just take a tour-bus information flyer and map and start hitching towards the same places. You can do small one-day excursions from central Göreme. Be sure to bring enough water though!
There are many abandoned caves like in Uşhisar (West side of the rock "castle" also known as the highest point of Cappadocia) and ürgüp (nice ones in the cliff over ürgüp kayak hôtel and fresco caves suite next to the Kefa küçük parc).