A comparison was made at DPReview of the new 24-105 f/4 with the 24-70 f/2.8L, the 35 f/1.4L, the 70-200 f/2.8L, and the 70-300 DO IS lenses.

Quick summary of results:

The 24-105 is not so good at 24, spectacular from 35-50 (even matching the prime), good at 70 (better than the 24-70), and not so good at 105. It's stronger overall than the 24-70 from 35-70, but weaker at the wide-angle side.

Given these results, my own personal thoughts are that the availability of IS on this lens still makes it a better choice than the 24-70 for general walkaround use, despite the high cost ($1,250 at B&H), and weakness on either end of the zoom range, and inability to reach f/2.8. It creates an excellent overlap on a full-frame camera with the Canon 16-35 and the Sigma 80-400. The 24-70 f/2.8 may still be better for indoor shooting.