In cold weather, it's the bag and extra clothing, for sure. I go pretty light with my shelter (single wall) and pack (frameless) because I've found I can sleep well and walk easily with those items. In summer, they're all about the same. Great comments above about consumables and body weight. Managing those items are very important.
Many hikers believe packing lighter will cost major bucks. There is a myth going around that for every one pound you loose from your pack, it will cost $100 extra. I've found the reverse to be true (with the exception of high quality down, but then you own high quality down which is always a good thing regardless of weight). Compare prices at Tarptent.com and Gossamergear.com. I've been very pleased with prices, service and quality from these "cottage industry" folks--they're avid hikers, too. I've consistently been able to find light weight stuff like rain gear and shoes on sale or in excellent used condition. My complete AT thru hike gear list weighed nine pounds cost me about $850 new (excluding shoes), for example.