Thought experiment, if a bunch of people went to live underground, would they evolve differently? Assuming they would live, die and reproduce the same as they did on the surface?

Like, does evolution only stem from survival of the fittest, or does our offsprings genetics change at all from the environment (outside of radiation and such of course)?