Pausing on Beacon Street in Boston to remember 32 Massachusetts workers killed on the job during 2012, including fishermen, drivers and painters, labor officials Thursday said penalties for negligent employers are inadequate and urged a tighter focus by policymakers on occupational dangers.

According to a report released Thursday by the Massachusetts AFL-CIO and the Massachusetts Coalition for Occupational Safety and Health, an estimated 320 workers in Massachusetts also died last year from occupational diseases, mostly associated with asbestos exposure.

Falls caused six of the worker deaths last year, motor vehicle accidents caused five deaths, and four fishermen died on the job last year. The average age of a worker who died on the job was 50.

Speaking on the steps of the State House, Catherine Devitt remembered her late husband, Geoffrey Almeida, a North Shore boat builder who died in 2011 after fighting mesothelioma, an asbestos-related cancer. Devitt said her husband was known among his co-workers for flagging workplace hazards and never would have knowingly exposed himself to asbestos, which she said should be banned.

“His death was terrible,” she said. “He did not die suddenly like so many, but the grief is the same. We lost him over a year and a half. He went from a vibrant 59-year-old to a 60-year-old man who could not live. He weighed very little. He looked like he was 90. At the very end, when he was told there was no longer any treatment for his illness, he asked the doctor if he got better could he come back for more chemo because that’s how much he wanted to live. He died a week later. Something must be done.”

The report’s release was part of Workers Memorial Day, which Massachusetts AFL-CIO President Steven Tolman described as a worldwide effort to remember workers hurt or killed on the job and to renew efforts to improve workplace safety.

Workplace fatalities were down last year. There were 58 on-the-job worker deaths in 2011 in Massachusetts, 47 in 2010, 62 in 2009, 66 in 2008, 80 in 2007 and 75 in 2006, according to the AFL-CIO.

“Every single workplace fatality represents an injustice that we should not tolerate and no worker should ever lose their life while simply trying to put food on the table for their family,” Tolman said.

Asserting that safety measures are often not taught or followed by many employers, Tolman said the average fine assessed on Bay State employers in connection with Occupational Safety and Health Administration violations resulting in a death of a worker was just under $10,000 last year.

“Inadequate penalties contribute to (an) employer mindset when they know it’s cheaper to violate OSHA regulations than to comply with them, as we all know at a great human loss,” he said.

Tolman also won a round of applause when he called on Congress to pass immigration reform legislation, saying 11 million undocumented immigrants “live and work in the shadows, often times in dangerous fields of work with no protections and no explanations.”