Quoting Fatty McRunnerPants:" Well at least she is getting to know her grandma. She has a right to know her dad's family regardless ... [snip!] ... It's what happens when you have a child with someone: you don't always get to make all the choices in how the baby is parented."
This. If you went out when you had your child and your Mom watched your child for you... you wouldn't complain, now would you? Unless he's abusing your child you really can't dictate what he does during that time. If he's neglecting her, sure, but his family has the right to see her and develop a relationship, and if he goes out... that's okay too. It really is.