Microsoft and other companies that thrived in the PC business have been scrambling to win back consumers who increasingly prefer smartphones and tablets.
Detractors say Ballmer contributed to the situation by not taking early threats from Apple and Google seriously enough. He consistently pooh-poohed Google as a one-trick company and in 2007 declared: "No chance that the iPhone is going to get any significant market share."
Ballmer's jeers proved premature. Google quickly made important inroads in Internet video, online maps, email and mobile computing. Those successes contributed to the damage that Apple's iPhone and iPad did to Microsoft and its partners in the PC market.
Although it derives some three-quarters of its revenue from sales of software and services to businesses, Microsoft has failed to capture the imagination of consumers who have become more enamoured with mobile gadgets. Response to the newest version of its flagship Windows operating system, Windows 8, has been lukewarm.
When Ballmer took the helm in January 2000, the company was worth more than $601 billion. Today, its value is less than half that amount, at nearly $270 billion.
"There is never a perfect time for this type of transition, but now is the right time," Ballmer, 57, said in a statement. He planned to stay on until a replacement is found. Microsoft said the search committee would include Gates.
After the news broke, Microsoft's stock shot up as much as 9 per cent and later came within two dollars of a 52-week high.
Ballmer's announcement comes less than two months after the company unveiled a sweeping reorganization of its business in an attempt to catch up with Apple and Google.
In his statement, Ballmer noted that Microsoft is moving in a new direction and needs a CEO that will be there for the longer term.
Microsoft, he added, "has all its best days ahead."
Ballmer met Gates in 1973 while they were living down a dormitory hall from each other at Harvard University. He joined Microsoft in 1980 to bring some business discipline and salesmanship to a company that had just landed a contract to supply an operating system for a personal computer that IBM would release in 1981.
Ballmer, a zealous executive prone to arm-waving and hollering, did the job so well that he would become Gates' sounding board and succeed him as CEO. He has worked at Microsoft for 33 years, matching the tenure of Gates, who left the company in 2008.
Though investors cheered the news on Friday, BGC financial analyst Colin Gillis cautioned that it could be a "tough 12 months" for the company.
The obvious successor — former Windows head Steven Sinofsky — got booted by Ballmer, he said.
Sinofsky left the company last year shortly after the launch of Windows 8. He recently announced that he joined the venture capital firm Andreessen Horowitz.
Veteran executive Julie Larson-Green, the head of Microsoft's devices and studios engineering group, has been floated as a potential successor. She was promoted to her most recent position in July, after being tapped in November to lead all Windows software and hardware engineering.
Although the company said Friday that it will consider both internal and external candidates, some analysts are betting that the company's next leader will come from outside.
When Ballmer joined Microsoft in 1980, it was populated with geeky programmers, led by Gates and the other founder, Paul Allen. Ballmer had already held a product-management job at Procter & Gamble and was attending Stanford University's graduate school of business when Gates convinced him to move to the Seattle area to whip Microsoft into shape.
Ballmer dropped out of Stanford, but only after Gates agreed to give him an 8.75 per cent stake in the then-tiny startup that had not even incorporated as a company. It turned out to be one of the world's greatest business partnerships.
By late 2012, Ballmer had accumulated an estimated fortune of nearly $16 billion from his initial Microsoft stake and additional stock options he later received.
He also was instrumental in growing Microsoft from a company that had fewer than 40 employees and $12 million in annual revenue when he came aboard. In 2012, Microsoft had 94,000 employees and $74 billion in annual revenue.
When he took to the stage to extol Microsoft, Ballmer often acted more like a crazed cheerleader than the chief executive of an influential company. In one presentation that eventually became a viral sensation on the Internet, Ballmer bounded across the stage, jumping up and down while yelping and imploring the audience to stand up, before breathlessly proclaiming, "I LOVE THIS COMPANY!"
But Microsoft enjoyed its greatest success with Gates at the helm and Ballmer as his sidekick.
Gates turned over the reins to Ballmer in January 2000 in what was considered to be a surprise move, because Ballmer had been considered more of a numbers and sales specialist, not a technology specialist.
The CEO change came just a few weeks after Microsoft's stock hit a record high of nearly $60, on a split-adjusted basis.
Janney Capital Markets analyst Yun Kim said investors shouldn't get too excited, because the company itself won't change overnight.
Kim said the new CEO, who will likely come from outside the company, faces the "daunting task" of making Windows relevant amid the continued consumer shift away from PCs.
Ted Schadler, an analyst at Forrester, said that while some may try to write its obituary, Microsoft still has some reliable cash cows. Its software, like Windows and Office, is still popular. So is Microsoft Enterprise, which helps big companies run databases, and the Xbox gaming system. Schadler noted that about 70 per cent of business email is still sent on Microsoft software.
Part of Microsoft's downfall stemmed from the bursting of a technology bubble that helped inflate the company's stock just before Ballmer took over.
But Microsoft also fell out of favour because many investors concluded that it was more interested in protecting its Windows franchise than coming up with new ideas and products to enter promising new markets.
By the time Ballmer took Google more seriously and began pouring money into a better Internet search engine, Microsoft already was hopelessly behind. The company's online division lost billions of dollars without putting a serious dent into Google's dominance of the field.
Google's rise riled the quick-tempered Ballmer, especially when key Microsoft engineers began defecting to the then-smaller company. After one Microsoft employee met with Ballmer in November 2004 to tell him he was leaving to join Google, Ballmer threw a chair across his office, according to a sworn declaration filed in a lawsuit. Ballmer then launched into an obscenity-laced tirade In which vowed to "kill" Google.
By 2012, the iPhone was generating more revenue than Microsoft was as an entire company and giving people less reason to replace their PCs. Again, Ballmer had to scramble to adapt and ordered a dramatic makeover of Windows so it could run on mobile devices as well.
The new system, Windows 8, borrowed many of its ideas from the software that ran the iPhone, just as Microsoft had copied some of the concepts for its early versions of Windows from Apple's Macintosh.
Meanwhile, the iPhone's immense popularity helped Apple overtake Microsoft as the world's most valuable company while Ballmer was CEO.
Microsoft also absorbed a $900 million charge to its most recent quarterly earnings to account for its expected losses from a company-produced Surface tablet that relies on a slimmed-down version of Windows 8 called RT.
Microsoft shares rose $2.36, or over 7 per cent, to close at $34.75. Over the past 52 weeks, the company's shares have traded between $26.26 and $36.43.
Associated Press writers Michael Liedtke in San Francisco and Christina Rexrode in New York contributed to this story.