It's a physics thing. The thing that keeps you warm in a sleeping bag is the tiny pockets of air trapped between the down feathers or the synthetic insulation fibers. When you compress those air pockets, the feathers or insulation itself isn't doing anything to keep you warm. You need the space. This is why you have to have a sleeping pad on all but the warmest nights--it's not just about comfort.
If you don't believe me, try sleeping without a sleeping pad some time and see if that compressed insulation underneath your sleeping bag is doing anything for you.