Associated Press technicians who witnessed the fall said one of the men appeared to have injured a leg and was taken away on a stretcher, while others sustained minor injuries.
The men were clearing snow at the Sanki Sliding Center, which will host the bobsled and luge events at next month's games in the Russian Black Sea city. The AP technicians said the workers appeared to have been harnessed to the building, but were swept down with the snow.
They estimated the roof was 10 metres high. The building is where the men's bobsled runs will start.
Security officers who arrived at the scene ordered the AP staffers to stop taking photographs with their phones, and briefly seized one of the devices.
Sochi organizers said five of the workers were taken to the hospital and were being treated for minor injuries.