It is that simple. I pretty much just focus on protein (for gains) and total calories and I’m in great shape. I do avoid sugary things (because it’s empty calories) but never had to care about oils or starches. If all you care about is not being fat you can just focus on calories.