Walking in the rain can wear you down if it goes on for awhile, say 2, 3, 4 days in a row.
A day of rain isn't a big deal. It can be kind of nice, actually.
On the AT there are shelters and ways to get away from it a bit, even if it's just to make dinner or something.
Best thing to do is be prepared and trust your rain gear/system and your tent/shelter, whatever it may be.