Pure math is build from the ground using axioms. ZFC is one axiomatization of set theory that has no contradictions, and that’s why it is so important within the math community. It basically allows you to build all of set theory, and if you include the axiom of choice, it also has a applications on different fields of math like analysis.

They’re just collections of elements and should be very simple, right?

If you haven’t, check out Russell’s paradox.