The full IOC formally ratified a decision made in October by the executive board that clears the way for Kosovo to compete as an independent nation.
Kosovo, which declared independence from Serbia in 2008, becomes the 205th national Olympic committee recognized by the IOC.
"This is one big day, one historical day for athletes of Kosovo," Besim Hasani, head of the Kosovo Olympic Committee, told the assembly. "This is the beginning of a new era for the Olympic movement in Kosovo."
IOC President Thomas Bach said: "This was a very important decision, particularly for the athletes of Kosovo. Now they can start to prepare for Rio."
Serbia has refused to accept Kosovo's independence and Olympic recognition had been fiercely opposed by Serbian officials.
But Hasani went out of his way in his speech to thank "my colleagues" from the Serbian Olympic Committee "for their friendship and all our international partners for their support."
Bach said Serbian Olympic Committee president Vlade Divac, the former NBA star, had not been happy with the initial IOC decision but accepted it "in the interest of the athletes" and in a gesture of sportsmanship.
"This is really a good example for the Olympic movement," Bach said.
Serbian Sports Minister Vanja Udovicic, however, attacked the IOC's decision as a "precedent" that could have "serious consequences for the global system of sports."
"What were the motives to accept the so-called European state that is not recognized by the United Nations?" Udovicic said in a statement. "And that is happening in a pre-Olympic year with the games taking place in Brazil, a country which did not recognize Kosovo."
Kosovo plans to send a team to the inaugural European Games in Baku, Azerbaijan, next year, and up to 10 athletes are expected to compete at the 2016 Olympics in Rio de Janeiro, including in the sports of judo, boxing and wrestling.
"We already have some very good athletes who in recent years achieved world gold medals and medals from European championships," Hasani said. "It's a good example of what can be achieved through sports and through autonomy and thanks to diplomacy of sports."
Kosovo's best known athlete, Majlinda Kelmendi, is a reigning judo world champion. She had hoped to represent Kosovo at the 2012 London Olympics, but competed for Albania instead. She has Kosovo and Albanian citizenship.
The IOC said Kosovo met the sports and technical requirements for acceptance, including the definition of a "country" in the Olympic Charter as "an independent state recognized by the international community."
Kosovo has not been accepted as a U.N. member. While more than 100 nations have recognized Kosovo, Russia and China have not. Most European Union members have also recognized Kosovo, with the exception of Spain, Greece, Cyprus, Romania and Slovakia.
The IOC said Kosovo's national Olympic body, set up in 1992, has more than 30 affiliated national sports federations, including 13 from Olympic sports. Six of those are full members of international federations.
Kosovo's top sports official, Memli Krasniqi, welcomed Tuesday's vote as a "truly historic moment."
"The IOC has finally ended two decades of injustice and isolation which has plagued Kosovo sport and which has prohibited the thousands of Kosovo athletes to be equal on the world stage," Krasniqi told The Associated Press in Pristina.
Associated Press writers Nebi Qena in Pristina, Kosovo, and Dusan Stojanovic in Belgrade contributed to this report.
Follow Stephen Wilson on Twitter: http://twitter.com/stevewilsonap