I think you would do yourself a big favor by getting a semen analysis done. Mine was relatively inexpensive, compared to the cost of testing hormones.
It very well may be your partner who has fertility issues, or you have just been unlucky in conceiving.
It could tell you whether you are completely infertile, or may just need more time to increase the odds than most guys.
A decent semen analysis will also test for white blood cells, which are a sign of infections like prostatitis.
There appears to be quite a few post-drug patients who went on to have healthy children, despite having sexual symptoms.