China's Jing Yu beat Nesbitt by just 0.02 points to take the title.
Nesbitt followed her world record in the 1,000 metres Saturday with a time just off that record to win the distance again Sunday.
But the sprint champion is determined by results in both the 500 and 1,000, skated twice for a total of four races. Times are converted into points.
Jing won on the strength of faster 500 metres, including a world record Sunday. Hong Zhang of China finished third, 1.10 points back of her teammate.
Jing skated the fastest 1,000 metres of her life to finish second to Nesbitt. Even though she was over half a second back of the Canadian, it was quick enough to claim the crown.
Nesbitt broke Cindy Klassen's six-year-old mark of one minute 13.11 seconds with a time of 1:12.68 on Saturday. Nesbitt went under 1:13 again Sunday, finishing in 1:12.94.
"I skated two world records and still didn't win, so it's disappointing," Nesbitt said. "I wanted to go under 1:13 again and I was happy I did that. Of course, now, I wish I had gone even faster."
Nesbitt was eighth and 10th in the 500 on Saturday and Sunday respectively.
Stefan Groothuis of the Netherlands won the men's sprint championship. He was the only man to skate under one minute seven seconds in the 1,000, which catapulted the Dutchman from fifth in the rankings to first for his first sprint title.
"At last," the 30-year-old said several times in Dutch before repeating it in English. "Pretty much before I got fourth.
"Finally I won a big championship and I'm really glad for that. Sprints is the most difficult one to win, so I'm really glad to win this one."
Kyou-Hyuk Lee finished second, 0.19 points back of Groothuis, and his South Korean teammate Tae-Bum Mo was third.
Jamie Gregg of Edmonton was ninth, followed by Muncef Ouardi of Quebec City in 10th. Denny Morrison of Fort St. John, B.C., was 13th.
The Olympic Oval in Calgary vies with its counterpart in Salt Lake City in claiming the fastest ice in the world. Two world records fell on the weekend. About 3,800 spectators watched the world championship over the two-day event.
The Oval is the training base for Nesbitt, a 26-year-old from London, Ont., who won an Olympic gold medal in the 1,000 in 2010.
Nesbitt and Jing were the final pair in the women's 1,000. Jing was still feeling buoyed from her world record of 36.94 in the 500, making her the first woman to go under 37 flat.
"I felt good about the 500 race, especially for the first 100 metres," Jing said through an interpreter. "No mistakes in the whole race."
Nesbitt crossed the finish line in the 1,000 wondering if she'd won back-to-back sprint championships after her victory last year in Heerenveen, Netherlands.
Disappointment flooded her face when it was calculated and announced that Jing took the crown.
"I wanted to defend my title from last year," Nesbitt said. "I wanted to win at home. I had really good one thousands, but my five hundreds weren't as strong as I wanted them to be."
Kaylin Irvine of Calgary was 23rd in the women's field, while Winnipeg's Shannon Rempel was 25th.
There are three world championships in long-track speed skating in addition to the World Cup circuit. Nesbitt will race in the world all-around championships Feb. 18-19 in Moscow in the 500, 1,500, 3,000 and 5,000 metres.
The world single-distance championships are March 22-25 in Heerenveen.