Do you hit pause for a longer rest?

I've been doing various FB workout programs for 10 weeks now and getting fitter with every day, but I always, without a fail, have to hit pause and take an extra 1-5 minutes depending on the intensity of the workout. Do you do the same or are 20 seconds enough for you?