Quoting Venus Penis Trap.:" <blockquote><b>Quoting Mama*AtoZ:</b>" Will his parents look after him well?"</blockquote> ... [snip!] ... Yeah, when I'm there they try to raise him! But if SO wants him to stay he needs to he the one to look after him."
hmm, could his parents be the ones behind him wanting your son to stay there with them?
i can see your preference is for your SO to care for him if he wants him to stay, but that doesn't necessarily mean it's a fact or that it's what's best for your son. What is your SO's POV? Maybe he doesn't like being seperated from his son either, and he might not feel confident looking after him which is why his parents take over.
On a seperate issue, if you're feeling uncomfortable with his parents then i think you should chat to your SO about it and both set some boundaries. He is your child, you are the parent and it sounds like it might become an issue that you should deal with now if you can.