I wouldn’t be. In the first place, your boys were preemies, and that means they shouldn’t be right on target for their developmental milestones. It takes a lot of years to make up for the time they missed in your uterus. In the second place, they’re twins, and twins tend to be slower in language development than singletons. Furthermore, they often develop a “private language” between one another that other people (even observant mothers like you) can’t translate.
I doubt that teaching them some signs slowed down their talking. In fact, I happen to think it’s an excellent idea and have urged teaching all children—and adults—a core set of signs that would enable people of whatever nationality to communicate at least minimally with one another.
It may well be that they have blended the signs you taught them into their own “twin language.” When twins do that, they often resist giving up the private language for standard English. Personally, I don’t think a formal speech evaluation is very important at their age. There are many 20-month-olds who don’t have any more language than you describe, and they aren’t twins and weren’t preemies. The therapist will undoubtedly have some good suggestions for you about how to stimulate their language, but I certainly wouldn’t want to put them in any sort of speech therapy at their age.
Our parenting advice is given as suggestions only. We recommend you also consult your healthcare provider, and urge you to contact them immediately if your question is urgent or about a medical condition.