First, at 17 years old, he was going to be drafted anyway, and he would rather, as a pilot, fly back to a bed after a mission than be part of the infantry that sleeps wherever the fighting is.
But more importantly, Ashby, an African-American man, wanted to prove a point.
"I was determined that we were going to prove we blacks can do anything anyone else can do," the now 89-year-old told The Early Edition's Rick Cluff. "We have the capabilities and we're going to prove that."
In December 1944, Ashby was sent to Tuskegee, Ala., to begin cadet training. He and a group of African-American pilots became known as the Tuskegee Airmen, the first black military aviators in the U.S.
On Friday, Ashby and other Tuskegee Airmen joined Chinese-Canadian war veterans and community leaders in Vancouver for the first annual Rights and Freedoms March, an event that that celebrates the Canadian Charter of Rights and Freedoms.
Friday's march holds significance for the Tuskegee Airmen who defended their country and fought for their rights despite facing racial discrimination that forced them to use separate entrances, washrooms and drinking fountains at their air base.
Like the Tuskegee Airman, Chinese-Canadians also served their country at a time of widespread discrimination and prejudice at home.
"The main thing individuals should realize [is they should] start thinking of other individuals as individuals themselves, not a particular group or race or anything of that type," Ashby said.
"This was something that we had to put up with throughout our early days in America with segregation going on."
The determination and demeanour of the original Tuskegee Airmen are what inspired people like Dick Toliver to join them.
"[Ashby] and the original airmen indeed have paved the way," said Tolliver, a second generation Tuskegee Airman.
"They made it possible for those who came along some 12 or 15 years later to believe that yes, if he could do it, then I could do it."
To hear the full story, listen to the audio labelled: Tuskegee Airmen march in Vancouver