Free speech is always perfectly fine. It is never an issue when a minority of people have extreme or weird positions. This is natural and unavoidable...

The thing the government actually has to watch for is the mob that comes to lynch those people or, even scarier, that comes to legally disenfranchise them and create laws that silence their political opponents.

Free speech is an implicit right to basic even to be given by the government... The first amendment shouldn't be about the government stopping free speech, it should be specifically about stopping the mobs and the politicians who inevitably come to torment the individual and create tyranny.