Love my Tilley Airflow. I wear is hiking every weekend (just day pack). I get asked a few times a year "what is that hat!!! Where can I get one!!". I backpack with it, and have not noticed the brim hitting the pack.
that said, I also have a UL, sun blocking, hiking cap with neck drape. I've yet to use it. I should try it.