Infertility, Secondary Support Group
"Secondary infertility" is difficulty conceiving after already having conceived and carried a normal pregnancy. Apart from various medical conditions (e.g. hormonal), this may come as a result of age and stress felt to provide a sibling for their first child.
It really bothers me that your husband would say that to you. I am so sorry, I know how that feels, my ex-husband was the same way-he always made little comments on me being a bad wife or mother because I did this or that. If you need to talk, I am here.