Quoting sταrry skies:" You can't know that he wouldn't use drugs around the children. It's always a possibility."
OP I know you don't want to hear it, but this is true.
My situation was very similar to yours. I can't give any advice because I left him, I couldn't deal with it. It was too much back and forth. It takes years to build trust back up. I wish you the best of luck but please, please, think of your kids first... if you think he's using in the house, while they're in his care, while he's driving them around, just leave and don't look back. This is a life long thing, like the previous poster said, not something you get over through a six month program or something.