I would say do every actual sidequest but don’t bother clearing the map of all question marks. Hunting for Witcher school gear is also just mostly cosmeticand optional, but they’re the coolest armors and swords.

Also, if you’re not playing on the lowest difficulty, read the infos in your journal regarding the creatures and prepare accordingly.