Uterus probs...

I'm 16 weeks and my uterus is measuring at the size of 12 weeks. I had an ultrasound and baby is growing fine is is perfect size- so its just my uterus. The doc said not to worry but at my next appointment, when I'm 20 weeks my uterus needs to be at least 18. But- I can't make it grow!

Why does this happen? What risks can there be if it doesn't catch up in growth?
Anyone know about this?