But in the case of Calgary residents Elise and Edwin Britton, an $18,000 tax bill from the Canada Revenue Agency, questioning their Canadian residency for tax purposes, came as quite a surprise.
The family went to China in 2004 after Edwin Britton accepted a two-year contract with an English training school there.
His wife and four dependent children went along with him and his contract was extended several times. Each time, the family decided to stay a little longer.
In the end, the Brittons spent nine and half years in China, filing tax returns every year to the Canadian government.
"[They were] accepted. Most of the time, I didn't make enough money there to have to pay taxes, except the last two years, I had to pay taxes.
"And they accepted my money graciously and so we had absolutely no reason to believe there was a problem," said Britton.
In fact, the family says they took things a step further to make sure there would be no problems on the tax front.
They say on the advice of a Canada Revenue Agency representative, they included a letter with every tax file, letting the CRA know where they were, what they were doing and when they'd be coming home.
Problem discovered by accident, says couple
But despite those efforts, just months before they returned to Canada for good, the family learned there was a problem—a problem that was discovered by accident.
In a phone call to the CRA to fix an issue with the Chinese address on file, Elise Britton says the tax agency started questioning whether the family should have been receiving the Canada child tax benefit and the GST/HST rebate for those years they were in China.
"When I spoke to the child tax benefit people, fear gripped my heart...when I started calculating that we could be owing them a whole whack of money."
The Brittons have nine children, some of them are grown, a few still live at home.
"[We felt] shock, fear...because we don't have [the money]. Anger because we really felt that we had been misled," said the couple.
After nine years of filing taxes, there was now a question as to whether the Brittons were residents or non-residents under Canada's tax laws.
While they were living outside the country temporarily, they believed they were "residents for tax purposes" as outlined in Canada's Income Tax Act. Someone can be a resident for tax purposes without physically being in the country.
"I don't understand why they suggested, in their correspondence with us, that we didn't inform them that we were in China and suggested that we were guilty of fraud for not doing it." said Edwin Britton.
"Because heavens, it's in nine cover letters that were included with our taxes."
While they were away, the family maintained its ties to Canada — returning to the country several times to see relatives, sometimes for weeks, sometimes months.
Edwin Britton remained part owner of a house in their hometown of Campbell River, B.C. Their children remained students of the Campbell River school system through its distance learning program.
The family also had Canadian drivers' licences, passports, bank accounts and personal belongings in storage for when they came back.
When the residency issue came up, Elise Britton sent proof of all these ties to the CRA.
"I was quite confident that given the paperwork, they would go, 'Oh yeah, you were residents for tax purposes' and that would be the end of it."
Not residents under the tax act
But it wasn't. The CRA ruled that despite those ties to Canada, and the family's intent to return,the Brittons were not residents under the tax act, and needed to pay back $18,000.
Riaz Mohamed is a chartered accountant who works with Calgary tax law firm Moodys Gartner.
"The case law on it is astronomical. This issue comes up all the time," said Mohamed.
Mohamed says when it come to determining status as a resident or nonresident for tax purposes, it's not as simple as whether or not someone is physically present.
There are three primary factors the CRA uses when determining someone's residential status while living or working in another country.
Under the Income Tax Act, Section 1:11:
The residential ties of an individual that will almost always be significant residential ties for the purpose of determining residence status are the individual's:- Dwelling place (or places).
- Spouse or common-law partner.
Secondary residential ties also come into play in the Income Tax Act. For example, if someone leaves personal belongings in Canada, has Canadian bank accounts, a driver's licence, or passport they could have a case for residency.
'Intent' an important part of case
And there's more, says Mohamed.
"You have to look at your whole life and a lot to do with that is your intentions. Are you planning on coming back to Canada?"
"In certain circumstances an individual can intend to come back to Canada but because of one reason or another, they can end up being outside of the country for a longer than expected time.
"But if their intention was still to return back — and they can prove that intention — then they should be factually a resident of Canada."
The Brittons not only intended to return, but they already have and are now living in Calgary. But that doesn't seem to help their case with the Canada Revenue Agency.
The CRA told the Brittons it looks at these sort of situations on a case-by-case basis.
In this case, the agency declined Go Public's request for an interview.
However, it offered information on how individuals can determine residence status before moving away.
That help comes too late for the Brittons. They admit they just can't afford to pay the $18,000 tax bill, but say this battle is not just about money.
"It seems to be so arbitrary and they don't answer to people. And so, that's wrong and I know that we're not the only ones that have had issues with the CRA and with unsatisfactory end results," said Elise Britton.
The family is now hoping to find resolution through the courts. Go Public has put the Brittons in touch with a University of Calgary program that takes on pro bono cases.
Submit your story ideas:
Go Public is an investigative news segment on CBC-TV, radio and the web.
We tell your stories and hold the powers that be accountable.
We want to hear from people across the country with stories they want to make public.
Submit your story ideas to Kathy Tomlinson at Go Public.
Follow @CBCGoPublic on Twitter.