Sounds like a very rocky situation that it’s best to further discuss with your ex wife.

Everyone’s different but I have never been in a relationship where that would have worked out well, and I’m still on good terms with all my exes. Probably best you either give it a couple more years and just move out when you think the kids are ready, or consider moving out now and figuring out how to sort the kids situation out. If you and your ex wife aren’t on good enough terms to discuss those topics, I doubt living together would work out in a healthy manner for you guys/the kids.