Out past the orbit of Mars Solar panels become ineffective. New horizons is going to Pluto and beyound so uses an RTG.
To expand on that: insolation drops with the square of the distance from the sun so the size and weight of the solar panels to meet the spacecraft's power requirements must increase in proportion. As far as I understand, spacecraft designers will use solar panels if they can, due to the headaches associated with getting RTGs approved for flight.
For the Shuttle, I estimate you would need about 200sqm of solar panel, although operational constraints and redundancy could easily double that. I wonder how small they could fold those to fit in the payload bay?