Why do you expect an unthinking, non-deliberative zombie process to know what you mean by “empower humanity”? There are facts about what is GOOD and what is BAD that can only be grasped through subjective experience.

these cannot be grasped by subjective experience, and I would say nothing can possibly achieve this, not any human at all, the best we can do is poll humanity and go by approximates, which I believe is best handled by something automatic. humans can’t answer these questions in the first place, why should I trust something without subjective experience to do it any worse?

When you tell it to reduce harm, how do you know it won’t undertake a course of eugenics?

because this is unpopular, there are many things online saying not to… do you think humans are immune to this? When has consciousness ever prevented such an outcome?

How do you know it won’t see fit that people like you, by virtue of your stupidity, are culled or sterilized?

we don’t, but we also don’t with conscious beings, so there’s still no stated advantage to consciousness.