This pretty much sums up my situation as well. My wife suffered a debilitating stroke in 2012 that limits her mobility, so we only do easy short hikes together
She has been my shuttle option on occasion when I want to do longer day hikes which allows her to be involved in my hiking plans.
I do believe that one can strike a mutual balance between family and hiking , both add joy and are equally rewarding.