Quoting Is that still my name?:" That's how I feel. His mom is a big part of her life. And she's a little manipulative as well.... I ... [snip!] ... I'm not... however.... if he is having harmful people around her, I damn well will keep him away... he's bad enough already."
When it comes to your child's well being, you have to put your foot down. If his mom tries to talk you into letting him see her, I would say "yeah, he can see her. He can come to visit her when she is with you, but he is not taking her overnight or alone at all. Period." If she oversteps boundaries...then unfortunately, you might have to cut her off til she can accept the rules you set in place.