It's probably better to go into a town without any preconceived notions about how you're going to be treated.
If you expect to be mistreated it could totally change your "vibe" and sure enough, you'll get mistreated.
Personally, I never met a town I didn't like. Manchester Center and Hanover were my 2 favorite towns and they're both a bit upscale from say... Erwin or Hiawassee..