To get every password, you’d have to exploit the password manager process itself. The manager asks you to approve every single password it hands out and you would know something is wrong if the extension starts asking for lots of passwords.

The separation keeps the memory where the passwords are stored away from the browser. No malicious code executing inside the browser can access it. Also, the protocol between the extension and the manager can be really simple and (hopefully) easy to get right without making exploitable mistakes.

It’s the Swiss cheese principle. The attacker has to break out of the website sandbox, get into the extension to copy the secret keys that are needed to impersonate the extension in the connection to the password manager, and exploit the password manager through that connection in order to get to the passwords. If any step fails (the holes in the cheese slices don’t align), the attack doesn’t get through.