The Thornhill, Ont., player seemed to lose control of his powerful serve in the second set, giving the ATP Tour's 38th-ranked player a chance to upset the world No. 6.
Chardy continued to outplay Raonic in the third, leading for most of the set.
But Raonic rediscovered his serve in time.
Chardy lost when his return of a Raonic serve bounced weakly before hitting the net on the decisive match point.