In question period Thursday, McCallum accused Stephen Harper of remaining silent on Williamson's comments, which were made last weekend.
He said Harper was giving in to "racist sentiments within his own party."
Williamson told delegates at a conference in Ottawa that it makes no sense to pay "whities" to stay home while companies bring in "brown people" as temporary foreign workers.
The New Brunswick Southwest MP later apologized "unreservedly" on Twitter for his "offensive and inappropriate language."
Williamson attended question period on Thursday, but Pierre Poilievre, the minister of employment and social development, responded to McCallum's comments instead.
Poilievre ignored McCallum's remarks and referred instead to complaints made by the Centre for Israel and Jewish Affairs about a speech by Justin Trudeau, in which the Liberal leader compared Canada’s immigration policy for Jews during the Holocaust to present-day fear-mongering about Muslims.
Williamson, a former director of communications for Harper, has called his comments about temporary workers the biggest mistake of his political career.
On Wednesday, New Brunswick member of the legislative assembly John Ames, also a Liberal, introduced a motion to condemn Williamson's comments and called them racially charged.
ALSO ON HUFFPOST: