The report shows that 36% of men and 40% of women would like to spend less time at work - even if that meant less money.
Less than 10% of those surveyed said they wanted to work longer hours.
British workers spend longer at work than their European counterparts.
4 million people regularly ignore the European Working Time Directive by putting in more than 48 hours a week, according to the Trades Union Congress and the Industrial Society.
While more bosses are getting round the rules by making employees take work home, as claimed by TUC.
Workers found it a lot easier to increase their hours than to reduce them, according to the survey by Essex University's Institute for Social Economic Research.
However, most employees do have some degree of control over the amount of hours they work.
Dr Mark Taylor, co-author of the study, said:
"But some jobs are more rigid in their hours, which may be caused by employer preferences, technology or industrial relations.
"What is more, the costs of changing jobs mean that some people persistently work more hours than they would prefer."
This suggests those in fear of losing work stay on and over work.
The survey found people increased the number of hours they worked when unemployment rates were high, suggesting fear was the main reason for the long hours culture.
Tired of living life with fear? Take action now and change your beliefs and life.