Well, I started backpacking in the Whites. Even in the summer, it can be cool up there. Then I moved to Colorado.
Basically, other than the middle section of the AT (and small portions of the PCT), never really had a reason to use a summer bag.
If I lived in the southeast Appalachians or the mid-Atlantic, a summer bag may be a good investment.
But not for me.