The buyer is Westbank Properties, which built the Shangri-La hotels in Toronto and Vancouver.
David Mirvish says the deal will close later this year.
He is the current owner of the property and son of the late Edwin (Honest Ed) Mirvish, who founded the store on the corner of Bloor and Bathurst streets in 1948.
David Mirvish told the Toronto Star that Westbank will rent the site and surrounding properties back to him for the next two to three years, while it decides what to do with the 1.8-hectare parcel.
The financial details of the sale were not disclosed, but there were reports prior to the sale that the asking price was $100 million.