He is scheduled to retire Wednesday and turn the Late Show over to comedian Stephen Colbert after more than 30 years as host.
Here is a look at some of Letterman's more memorable moments.
1st show after 9/11 attacks
Letterman opened his first show after the Sept. 11, 2001, attacks with a heartfelt monologue heaping praise on New York's then-mayor Rudy Giuliani and its firefighters and police. Visibly emotional, he also described a small town in Montana with a population of 1,600 that had recently donated money to the City of New York.
"If we are going to continue to do shows, I just need to hear myself talk for a couple of minutes," he said during the taping of the Sept, 17, 2001, show.
Musician Warren Zevon was the featured guest for an entire episode that aired on Oct. 30, 2002, after he had been diagnosed with cancer. He performed songs and talked with Letterman about his life and how things had changed.
"You're reminded to enjoy every sandwich, and every minute of playing with the guys, and being with the kids and everything," Zevon said. He died less than a year after the interview.
Admits to having sex with colleagues
Letterman admitted to having sexual relationships with some female staffers during his show on Oct. 1, 2009, after a man threatened to publicize the information if Letterman didn't pay him $2 million. The host spent 10 minutes outlining how the scenario unfolded, which he said had been a "very bizarre experience."
He later apologized to his current female colleagues, saying journalists had been asking if they were in a relationship with Letterman when the events he was describing happened in the past.
The host walked off his own set during an uncomfortable appearance by Crispin Glover on July 28, 1987. The actor was behaving erratically, rambling about the press and challenging Letterman to an arm-wrestle before kicking the air near the host's face.
Letterman stood up and walked away saying he had to check on the Top 10 list.
"He came very close to denting my head with those giant shoes, so I thought: I don't need that," he said later in the show while explaining why Glover had left. "I'm 40, I went to college."
It has been speculated since that Glover, known for his performance as George McFly in Back to the Future, was performing "in character" to promote a new movie, Rubin and Ed. The actor himself has never confirmed it was an act.
Madonna and Letterman traded barbs with each other during a memorable interview in 1994. Madonna said he had changed since she was first on the show, suggesting he kisses up to his guests. While smoking a cigar, she also repeatedly cursed as an exasperated Letterman tried to get her to stop.
"Will you stop it? Ladies and gentleman, turn down your volume," he said.
The actor gave a downright unusual interview on the Late Show in February 2009. Sporting long, unkempt hair and a bushy beard, Phoenix mumbled throughout an interview, as Letterman joked about his appearance and an alleged foray into hip-hop music.
Phoenix later apologized and admitted he was putting on an act for an upcoming film, I'm Still Here, but it still makes for an entertaining watch.
John McCain ditches Letterman
John McCain became the target of Letterman's jokes after cancelling an appearance on the Late Show at the last minute during his presidential bid in 2008. The senator said he had to return to Washington to help fix the economy.
Although he hailed McCain as a war hero, Letterman poked fun at his campaign and lambasted the alleged experience of his running mate Sarah Palin. Letterman later showed footage of McCain setting up for an interview with CBS News's Katie Couric at the same time he was supposed to be on his show.
Sarah Palin apology
Letterman apologized after a sexual joke about Sarah Palin's daughter prompted outrage from her family. He had suggested Palin's daughter had been impregnated by baseball player Alex Rodriquez while she was in New York.
He said the joke was about her 18-year-old daughter Bristol, but in fact her 14-year-old daughter had actually attended the game. He said the confusion was his fault and that the joke was "beyond flawed."
Drew Barrymore's birthday surprise
A very energetic Drew Barrymore seemed to shock Letterman with merely mentioning that she had been to a "nude performance dance club," during a 1995 interview that happened to be on his birthday.
He seemed downright flabbergasted after she asked his house band for some blues, jumped up on his desk and started dancing. Then she lifted up her shirt before getting down and resuming the interview.
Letterman had comedian Andy Kaufman and wrestler Jerry Lawler come on the during a public (and staged) feud between the two in 1982.
Kaufman demanded an apology for alleged injuries he suffered while the two wrestled in Memphis, Tenn.
Lawyer stood up as the show was going to a break and appeared to slapped Kaufman hard enough that he fell out of his chair. The comedian later returned and unleashed an expletive-filled tirade before tossing coffee on the wrestler and high-tailing it out of there.
It may have been faked but it was certainly entertaining.