The CFIB's latest confidence barometer fell 3.5 points to 63.5 per cent in June, erasing the gains seen the previous month.
The business group says a reading of between 65 and 70 is indicative of an economy growing at its potential, so the result suggests there is still some skepticism about the strength of the recovery.
The result was not all bad news however as the drop reflected a "darker mood" mostly in the hospitality and construction sectors. New orders remained strong, concern about lack of local demand fell to its lowest levels this business cycle, and short-term hiring plans were reasonably good for this time year, the CFIB said.
Canada's economy has gone through a rough patch of late since the strong fourth quarter of last year, with growth falling to only 1.2 per cent in the first three months of this year. Meanwhile, the first quarter in the U.S. saw the economy there slide 2.9 per cent due to the unusually harsh winter.
But many analysts believe there is more to Canada's flat performance so far in 2014 than bad weather. The long-anticipated recovery in the export sector has yet to materialize and business investment has been disappointing. Meanwhile, the domestic economy, has cooled as expected.
In a note to clients, Scotiabank economists Derek Holt and Dov Zigler said Canadians shouldn't expect a big pick-up in the second quarter either, although it will likely be better than the first.
"The Canadian economy put in a very mixed performance to start the second quarter, and so we're playing it cautiously with respect to a call for modest growth when Monday's monthly GDP print for April lands," they wrote. They estimated that second-quarter growth in Canada will likely come in at roughly two per cent.
Consumers also appear cautious, according to a survey from the Chartered Professional Accountants of Canada that suggests Canadians won't be swarming to the malls this summer.
The survey found only a quarter of Canadians saying they intend to spend more this summer than last, suggesting households are in a saving mood. About the same number plan to spend less. The survey found similar results on vacation plans.
Regionally, the drop in optimism among smaller businesses was most acute in Ontario, Manitoba and Saskatchewan.
The June CFIB survey was based on 1,042 responses from CFIB members received through June 16.