The other option, if you don't overcome your fear - is something like Gymboree classes, you know, the music and fun or whatever they're called, other 'mommy and me' events that don't entail him being alone.
Hit the local parks on a regular basis.. make friends, organize playdates.
On the other hand - he will be starting school sooner or later and unless you plan on homeschooling him, you're going to have to let him go at one point or other, just right now you have the option not to yet
There are other options out there for letting him interact with other kids. Do you know many other moms in the area? You could try to set up some park days. Or see if there are any classes that you two could do together - tumbling, swimming lessons, etc etc.