He was past his 30th birthday, they would point out.
About 2 1/2 years went by without any additions to his Grand Slam trophy case, the thinking went.
First Rafael Nadal, then Novak Djokovic, overtook Federer in the rankings and as the man to beat at major tournament after major tournament.
Well, look at the guy now. Wimbledon champion, once again, stretching his record total to 17 Grand Slam championships. Ranked No. 1, once again. And — heading into Monday's start of the U.S. Open — the favourite to reach the final, once again.
"I'm out of the business of predicting Federer anymore," said Andre Agassi, a two-time U.S. Open champion and runner-up to Federer in 2005. "He's continually surprised me with his achievements; he no longer surprises me. I think he has a lot more tennis in him. He looked as comfortable as I've ever seen him on the tennis court in England. He maybe needs one or two things to fall for him to knock down a few more (Grand Slam titles) at this stage of his career, but he's certainly as capable of it as anybody I've ever seen."
Federer's pursuit of a sixth U.S. Open title at age 31 will certainly be among the main angles to keep track of on the hard courts of Flushing Meadows.
Other stories to watch include:
— Djokovic's bid for a second consecutive championship in New York and fifth major title in two years;
— Andy Murray's attempt to follow up his Olympic gold medal with Britain's first Grand Slam men's singles title since 1936;
— Andy Roddick's hope for one more deep run in front of the home fans;
— Four-time major champion Kim Clijsters' farewell to tennis in what she says is the last tournament of her career;
— Venus Williams' return to the U.S. Open a year after withdrawing from the tournament and revealing she was diagnosed with an autoimmune disease;
— Serena Williams' try for her 15th major trophy — and, of course, what sort of interaction she might have with on-court officials after a foot-fault tirade in the 2009 semifinals, then a "you're just unattractive inside" monologue in the 2011 final.
"My mind frame this year is that something is going to happen, for sure, because something always happens to me at the Open, whether it's a horrendous line call that's 2 feet in or whether it's a grunt and I get a point penalized or a foot-fault when I actually don't foot-fault. I'm prepared for something to happen," said the younger Williams sister, a three-time champion in New York whose serve was dominant recently en route to her fifth title at Wimbledon and two gold medals at the Olympics.
Another key question is what sort of effect there will be from the short turnaround and shift to hard courts after the grass-court London Games.
"There's no doubt about it: This is not an ideal preparation," said Federer, routed 6-2, 6-1, 6-4 by Murray in the Olympic final on Aug. 5, less than a month after beating him in four sets on the same court in the Wimbledon final.
"It's not impossible," Federer added, "but it's just very hard on the body and mind to travel halfway around the world, go on a different surface. ... In the past, you would take maybe a few weeks off for a top player, then prepare for three brutal weeks on hard courts, then come over here wanting to fire (on) all cylinders. This year, it's different."
He once won 40 matches in a row at the U.S. Open, a streak that ended with a five-set loss to Juan Martin del Potro in the 2009 final. That was followed by semifinal setbacks against Djokovic each of the past two years, including what Federer calls "that brutal match with Novak" — in 2011, when Federer took the first two sets, then held two match points, but couldn't close the deal.
On the first match point, Djokovic smacked a gutsy forehand return winner that barely landed on a line and drew something of a rebuke from Federer afterward.
"I never played that way," Federer said at the time. "I believe in the hard-work's-going-to-pay-off kind of thing, because early on, maybe I didn't always work at my hardest. So for me, this is very hard to understand how (he could) play a shot like that on match point."
Djokovic went on to victory in the final against Nadal, the 2010 U.S. Open champion and 11-time major winner who is currently out with knee problems and won't be in New York. That was part of a stretch in which the Serb and the Spaniard split up nine Grand Slam trophies in a row, shutting out Federer and leaving him stuck on No. 16 for more than two full seasons.
Given how rare it is for a man past 30 to remain in the upper echelon of tennis — a not-quite-31 Federer was the oldest Wimbledon champion since Arthur Ashe in 1975 — and the ascension of a couple of rivals in their mid-20s, there were plenty of whispers that the Swiss superstar might be done.
A man who reached a record 10 consecutive major finals from 2005-07, then another eight in a row from 2008-10, suddenly was losing the occasional Grand Slam — gasp! — quarterfinal.
What, after all, was left to motivate him?
What could drive him to keep up?
Djokovic and Nadal, to name two, never doubted Federer would reassert himself.
"He was always there," Djokovic said. "Last couple years, he didn't win a major, but he was in a couple of finals and always semifinals. He's always playing close matches against whoever."
Federer beat Djokovic in the Wimbledon semifinals, then topped Murray for triumph No. 7 at the All England Club, tying Pete Sampras and William Renshaw (who played in the 1880s) for the most in history. That also allowed Federer to return to No. 1 and, a week later, break Sampras' career record for most time atop the ATP rankings.
Continuing what he termed "a magical summer for me," Federer earned his first individual Olympic medal. Then he showed he can still turn up big on hard courts, winning a record-equaling 21st career Masters title last weekend, holding serve throughout the tournament and beating Djokovic in the final.
Federer, Djokovic and Nadal have combined to win 29 of the past 30 Grand Slam titles (del Potro is the only interloper in that span, which began in 2005).
Murray has been on the outside looking in, but there are suspicions that his success at the Olympics could be a harbinger of what's to come.
"Come the U.S. Open, I hope this will have given me the confidence to go there and believe in myself a bit more than I have in the past," Murray said at the Olympics, "and give myself a shot at winning there."
It also means he already owns one of this season's top five prizes; Djokovic won the Australian Open in January, and Nadal won the French Open in June, before Federer re-emerged at Wimbledon.
"It is interesting, obviously, that three different guys have won three different majors this year, plus Andy the gold," Federer said. "It definitely sets a great tone for the U.S. Open, there's no doubt about that."
With Nadal sidelined, and Murray still waiting to win a major final, Federer and Djokovic appear set to take centre stage at the U.S. Open.
At the very least, Federer is firmly back at the forefront.
"Putting my pure fan hat on: He's one of the greatest of all time. It's not only the quality of his play; it's how he represents our sport on the court and off the court," WTA Chairman and CEO Stacey Allaster said. "He's just one of those athletes that we'll all look back and say how blessed we were to have been able to see him perform at the highest level of our sport that we've ever seen, and for such a long period of time."
Follow Howard Fendrich on Twitter at http://twitter.com/HowardFendrich
AP Sports Writers Rachel Cohen, Rick Gano and Steven Wine contributed to this report.