Quoting Super Kame Vegeta:" You guys are right. I needed an outside perspective to all of this. My kids are fed, clothed, and ... [snip!] ... away from me he'll have to fight me. And he's got records of his problems with mental health, so there is no way he'd win."
don't believe for 1 second that you're not strong enough to do this! You are and he knows it which is why he's trying to manipulate you with your 1 weak spot (your kids). Any mom wants the best for her kids and he's trying to make you feel like you can't give them that, but you can. As long as you love them and can feed and clothe them, you will all be fine. The sense of security that comes along with being stable with your parent is worth more than $, I'm not saying don't let them see him but keep them together and stay strong!