The first step here is not to kick him out. It's to go to the court house and apply for custody before he does. In a case where custody is not established, either parent can go to the court and get it, no questions asked, at least temporarily. So before you piss him off, go get it documented that you have legal custody.
No joke, my brothers father got an ex parte custody order this way when my brother was sixteen!
I don't know about in ur state. You will need to research it but In my stae if u have a child out of wedlock & are still unmarried to the child father he cannot take ur baby. If he does he will get a felony charge for kidnapping. Reason is because (in my state at least) A child born out of wedlock the custody is atomatically granted to the mother. Now also if ur childs father does run off with ur baby not only will he get that lovely felony kiddnaping charge but that also qualifys as custodial interferance. Meaning that if he does it & you have proof then that is grounds to take him for full custody.. Again i say all this because that's how tennessee is. I know because i went through it. It was awful my BD had my daughter for 2 weeks before i got it all figured out & the cops took me to his house to get my daughter in his county after I had to have a detective in my county print me the papers out from the law book. Some countys around here are not aware of that law. So I wish you the best of luck with all of this. DO the research. And don't stop looking until you get the answers you need. It took me two weeks.
Oh yes and also i need to tell you that if something was to happen & him take ur kid & him be made to bring it back he cant try to file for emergency custody where they will go check out where ur living & drug test you. So make sure you got your stuff in order.