The groups say they wrote to Raitt last month seeking a meeting with her in Ottawa for the last week in February, but she has not responded.
Lynn Jacobson, president of the Alberta Federation of Agriculture, says farmers want to know why Ottawa can't get railways to ship more of the bumper crop to port for export to customers around the world.
"The transport minister has to get involved in this," he said Tuesday.
"This is a crisis situation and something has to be done. It affects not only the agriculture community but the whole economy in Western Canada."
The federal government is aware that grain farmers have been watching the quality and value of their crops decline over the last few months in fields or in storage because of the transportation bottleneck.
Agricultural groups say they warned Ottawa about the potential of the backlog months ago and haven't been shy about lobbying for change.
Doug Chorney, president of Manitoba's Keystone Agricultural Producers, recently wrote an opinion article that has been published in some Prairie newspapers that calls on Ottawa to take action sooner rather than later to fix the bottleneck.
The article lays the blame on "abysmal service" by Canada's two major railways.
It says the monopoly Canadian Pacific Railway (TSX:CP) and Canadian National Railway (TSX:CNR) have in the marketplace allows them to provide inadequate service without fear of consequences.
"We need short-term intervention by Transport Minister Lisa Raitt and the federal government because farmers need to get their crops to market as soon as possible before more sales are lost," wrote Chorney, who farms near East Selkirk, Man.
"This is not a once-in-a lifetime situation. Instead, it will be the norm in a few years. Will we wait until then to fix our rail system, or will we begin to do it now?"
Norm Hall, president of the Agricultural Producers Association of Saskatchewan, said his group wants to tell Raitt face to face that the Fair Rail Freight Service Act passed by the Conservative government last year isn't working and should be amended.
Transport Canada has said the law provides incentives for grain companies and railways to voluntarily negotiate shipping service agreements.
Hall said to his knowledge no grain shipping companies have signed contracts with the railways.
"Put some responsibility on the railroads," Hall said. "If there is inaction by the railroads, there need to be penalties."
Roxane Marchand, a Transport Canada spokeswoman, said in an email that the government continues to monitor the effectiveness of the freight service legislation.
She said the record fall harvest has put significant pressure on grain-handling and transportation.
The department is working to establish a national forum for companies and producers who ship commodities by rail to deal with problems and make improvements, Marchand said.
"Transport Canada expects to launch the commodity supply chain table in the coming months," she wrote in the email.
The department did not respond to questions about whether Raitt will meet with the farm groups in Ottawa at the end of the month.
On Monday, Agriculture Minister Gerry Ritz announced that Ottawa plans to keep closer tabs on rail companies by requiring them to report monthly on their performance instead of every three months.
The federal government is also spending $1.5 million for a five-year transportation study.
Hall said action is needed this year or the problem could continue next fall and into the future.
"We can't wait that long. We could have the same glut of grain on the Prairies next year."
CN spokesman Mark Hallman said the railway has been performing at a record pace with grain car shipments about 12 per cent higher than the five-year average.
He said cold weather after the harvest, especially the "polar vortex," has taken its toll on CN's performance.
"Extreme cold weather starting in early December became an issue ... hampering CN operations and leading to a downturn in transportation performance for all commodities, including grain," Hallman wrote in an email.
"Very cold weather continues to affect CN's operations in Western Canada."
Hallman said CN uses shorter trains during cold weather to ensure breaks can be used properly. Shorter trains mean more of them are needed, necessitating more crews, which affects shipment capacity.
CN plans to significantly improve the speed and reliability of all commodity shipments, including grain, once the weather improves, Hallman said.