Getting back into exercise

Over the past few months I had partial improvement in my symptoms but I’m still far from 100%, my post-fin baseline is pretty good, but I feel like I’m on thin ice as minor things can bring me crashing back to square one.
Most notably, last time I did a chest workout back in June my anxiety went right up and ED went from mild to severe, following this temporary crash it took me about a week to get back to my regular post-fin baseline.
I’m not sure why this happened, perhaps I went too fast too soon, but from what I’ve seen some other guys have had a similar experience. But anyway, I have been pretty idle since this whole fiasco happened and really want to get back into working out, but I’m really scared about this happening again.
Why might this be happening? May I should just try and slowly ease myself back into things, but still, I’m too scared to take the risk now.