Terminating parental rights

I know I need to speak with a lawyer, I'm just trying to figure out if I'd even have a case.
I found out my daughter's dad was doing heroin when I was 4 months pregnant with her. I tried to help him through countless rehabs and treatment but nothing ever worked. When she was a month old, I thought he was clean since he was in an outpatient program and seemed to be doing alright. She woke up to eat one night and I started crying, asking him to please feed her. He hadn't helped out with much of anything since she was born except maybe 1 or 2 diapers and I was exhausted. I woke up an hour later to him passed out in the rocking chair and her on the floor. I tried waking him up and I did slap him because he didn't even realize what he had done. I kicked him out as soon as he woke up the next day. I then found out he had drained my entire savings account and had pawned damn near everything I owned, including things for my daughter. Long story short, he ended up in prison not too long after for a few different things. His brother (the only person I stay in contact with from that family) also just told me a few days ago that he had some other charge for flashing his gun at someone for cutting him off while driving.
He was in prison a year and a half and when he got out, after discussing with my fiance, we decided to let him see my daughter, who is now almost 3. Our custody agreement from our divorce stated that he couldn't see her until he was off parole and tested clean for a year after that. He had never fought the divorce so it never even went to court. I didn't really have any intention of keeping my daughter from him if he was doing well, I just asked for that to protect my daughter if I didn't think she needed to be around him. She saw him a few times at parks and stuff but it started getting difficult for me to find time to leave and allow them to visit. My fiance and I let him come over one day since I really didn't have time to leave during school. My daughter had started going back and forth between calling my fiance by both his name and daddy. I never stop her from calling him whatever she wants. And in all honesty, he is her father in every other sense of the word except actually creating her. Well my ex overheard her call him daddy and blew up, trying to fight him in front of my daughter. I told him he needed to leave and said that was it for them visiting and that we would be going back to what the custody agreement says. He has tried texting me once since then to see her, but I ignored it.
He gave me 1200 while he was in restitution that he wanted me to hold until he got out since they kept taking money from him. He ended up telling me to just keep it in the end though. The majority went to Christmas gifts, and the rest to help pay off some bills. This is the only money I have ever received from him. The divorce asked for $100 a month in CS since that was the minimum here, but I've never enforced it. I know based on what he's making now, he should be paying $500-600 a month. He has never once offered to start paying anything though and it's just seemed like an unnecessary headache for me to fight when I'm doing fine without it.
When he got out of restitution, he moved his mom in with him. His mom is an alcoholic and ex drug user as well. She's even crazier than him. But from what I understand, they're starting to fight now. His brother told me that his niece has been up here the last 2 weeks for break and called her other grandma last night crying because they were in each others' faces, fighting so badly. That is NOT something I ever want my daughter around.
My question is, would I even have a case? My daughter has no idea who he is when he comes around. She thinks he's her friend. She has a stable home now and really is not missing out on anything by not knowing him. I've also heard that in order to take rights from one parent, there has to be someone else willing to take over, but I'm not sure how true that is. My biggest worry is that if something were to happen to me, he'd end up getting her if he still has rights. The thought of that terrifies me. Thanks for reading and for any insight you can offer!

Typically a man's rights cannot be terminated unless you're married, and your spouse is willing to legally adopt.

In an ideal world, hopefully him and his family get their smurf together so your daughter can have a healthy and meaningful relationship with them.

Good luck!

1. Why are you not enforcing child support? You can't just sit on your hands while you complain that he isn't doing anything....you have the power to have his wages garnished.

2. No, I truly doubt he would have rights taken away. Unless you are married to someone else who is willing to adopt the child, I truly doubt that it would happen.

In kentucky you have to be marrie dover a year before he can adopt her and he has to agree to give his rights up unless he is a harm to the child. i know this because i am going threw the same process. why not get supervised visits or something? in your case it sounds better for him to just be out of the picture all together good luck.

Quoting speaktruth2powr:" 1. Why are you not enforcing child support? You can't just sit on your hands while you complain that ... [snip!] ... taken away. Unless you are married to someone else who is willing to adopt the child, I truly doubt that it would happen."

1. That's not really fair, there is an agreement already from what she said. He should be willing to do it as the parent and shouldn't want to make a bad situation worse by court orders. Instead he's not. She's not to blame for him not wanting to pay for his own child.

I'm not sure about the law, but I have heard you have to be married as well.

Quoting speaktruth2powr:" 1. Why are you not enforcing child support? You can't just sit on your hands while you complain that ... [snip!] ... taken away. Unless you are married to someone else who is willing to adopt the child, I truly doubt that it would happen."

Definitely!

Even if you don't "need" the money, OP, you can put CS payments in a savings account for college or something.

Quoting Your friendly n00berhood :" 1. That's not really fair, there is an agreement already from what she said. He should be willing to ... [snip!] ... for him not wanting to pay for his own child. I'm not sure about the law, but I have heard you have to be married as well."

She has stated several times that "she hasn't enforced it".....so, why not enforce it if sitting on her ass doing nothing isn't getting her very far?

Quoting speaktruth2powr:" She has stated several times that "she hasn't enforced it".....so, why not enforce it if sitting on her ass doing nothing isn't getting her very far?"

Her "not enforcing it" doesn't mean that the courts haven't ordered it. It just irritates me a little when mothers are called lazy or told they're "sitting on their ass" when it is the fathers responsibility as the father to support his child.

Quoting speaktruth2powr:" She has stated several times that "she hasn't enforced it".....so, why not enforce it if sitting on her ass doing nothing isn't getting her very far?"

Sitting on my ass? I have been a single mom, held a full time job and been a full time student while supporting my daughter 100% on my own. I have not enforced it because, like I said, it is more headache than it has been worth. I am not asking about the child support. I know how to go about getting it, I just don't need it. My question was about getting his rights taken since he really hasn't cared to be involved with anything to do with her.

Quoting Your friendly n00berhood :" Her "not enforcing it" doesn't mean that the courts haven't ordered it. It just irritates me a little ... [snip!] ... called lazy or told they're "sitting on their ass" when it is the fathers responsibility as the father to support his child."

Thank you. It has nothing to do with me being lazy.

Quoting Krystal +2 Boys:" In kentucky you have to be marrie dover a year before he can adopt her and he has to agree to give his ... [snip!] ... get supervised visits or something? in your case it sounds better for him to just be out of the picture all together good luck."

Thank you! SO and I are engaged but I wasn't really in any hurry to get married. He is more than willing to adopt her ASAP though. My ex's brother told him to just give them up. That he really isn't her father. But who knows if that'll happen.

Quoting Audra's mommy:" Sitting on my ass? I have been a single mom, held a full time job and been a full time student while ... [snip!] ... it. My question was about getting his rights taken since he really hasn't cared to be involved with anything to do with her."

Considering that a large part of your post is complaining about how he doesn't pay you anything for child support....I'm thinking it does have something to do with it.

All it takes is a phone call to Child Support Enforcement....

Quoting Audra's mommy:" Thank you. It has nothing to do with me being lazy."

Your best bet is to just talk to a lawyer and see how it works in your state they are still letting BD terminate his and i dont have anyone to adopt. people on here are really touchy on this subject

Quoting speaktruth2powr:" Considering that a large part of your post is complaining about how he doesn't pay you anything for ... [snip!] ... support....I'm thinking it does have something to do with it. All it takes is a phone call to Child Support Enforcement...."

That was not me complaining. I only mentioned it because I don't know if that even matters in court. But thanks for your help!

Quoting Audra's mommy:" Thank you! SO and I are engaged but I wasn't really in any hurry to get married. He is more than willing ... [snip!] ... ASAP though. My ex's brother told him to just give them up. That he really isn't her father. But who knows if that'll happen."

He can't just "give them up"....it doesn't work that way.

And you and your fiance have to be married for a set amount of time before he adopts her. Depending on your state it is anywhere from 12 to 24 months.