The inside workers union, CUPE Local 79, has scheduled votes for the 23,000 employees on Tuesday.
CUPE has accused the city of bargaining in bad faith and filed a complaint with the Ontario Labour Relations Board.
Inside workers include civil servants, child care workers, ambulance dispatchers, nurses and janitors.
The two sides will be in a legal strike-lockout position Saturday.
Local 79 president Tim Maguire said the city is asking for major concessions and a deal that offers much less than the one negotiated for outside workers, who settled in February.
“We did not come by the decision to seek a strike mandate lightly,” Maguire said in a statement released Sunday.
“We are not holding a vote because we want to stop working. We are doing it because we want the city to start talking.”
The library workers called a strike Sunday after negotiations, which had been extended several times during the day, failed to produce an agreement.