Depends on the town. I grew up in and went to college in a small town - pop. 16,000 ish. It was the reverse, students came for mostly 3 reasons; it was as far from home and still pay in state tuition, it was (is?) a quirky place where no one really cared how high you flew your freak flag, or to for the pot.

The college brought in diversity and employment, but wasn’t really the identity of the place.