Working in hot and potentially humid conditions creates health and well-being risks that will increase as the planet warms. It has been proposed that workers could adapt to increasing temperatures by moving labor from midday to cooler hours. Here, we use reanalysis data to show that in the current climate approximately 30% of global heavy labor losses in the workday could be recovered by moving labor from the hottest hours of the day. However, we show that this particular workshift adaptation potential is lost at a rate of about 2% per degree of global warming as early morning heat exposure rises to unsafe levels for continuous work, with worker productivity losses accelerating under higher warming levels. These findings emphasize the importance of finding alternative adaptation mechanisms to keep workers safe, as well as the importance of limiting global warming.