Upper back pain/stiffness

I enjoy the FB Strength program a lot, seems I've gain some muscles (I feel bigger) but with small issue. I work behind PC whole day, sometimes more than 10 hours a day and always after training of upper body (especially after overhead press) my back is stiff and sometimes even painful.

I am not trying to train through pain so I'd like to ask you what kind of stretches can I do, that would reduce the stiffness/pain without warming and cool down (basically behind desk) and how often should I do it?

My doctor recommended me swimming, but that is not an option right now.

Thank you upfront!

Edited