Scared of losing muscle mass

I know this may sound silly to some, but for the past few months I've really wrestled with the fear that, if I take a week or two off, I'll lose all the muscle I've built and progress I've made.

It's become hard to deal with, because on the one end exercise has become sort of a ''safe haven'' for me, but on the other I've noticed that I've been more prone to injury, fatigue, and general bodily destress when working out!

Is there anything I can tell myself or do that'll help me take a step back and not worry about losing my gains?