Canadian Medical Students Demand Change After ‘Inhumane’ Exam Conditions

Dr. Brianna Barsanti-Innes was about five hours into her seven-and-a-half hour digital medical licensing exam when the system disconnected.

The morning of June 12, she’d launched the ProProctor app on her laptop — the digital platform from Prometric, the American-based company that administers and proctors the exam — and, using her laptop’s camera, did a security check to show she wasn’t hiding any recording devices. The University of Toronto graduate ensured her partner was out of the apartment, because you’re not allowed to have anyone else there.

After her lunch break almost midway through the exam, she couldn’t connect to a proctor. When she eventually did, she was told her camera wasn’t working and that she should log off and relaunch the app. After waiting almost an hour without being able to connect to a new proctor, she grabbed her phone to check a med student Facebook group and saw that several others were experiencing similar issues.

She called Prometric’s global help line, and waited another half an hour to get through to someone. She was told there had been a technical error, and her exam was done for the day. A proctor then came on her laptop screen and told her the same message.

“We were all a little thrown by this,” Barsanti-Innes told HuffPost Canada. “I was 80 per cent done the exam, people were 95 per cent done, some people were in the process of literally looking over their answers before they submitted the exam.”

She was one of 33 students out of 67 writing with remote proctoring through Prometric’s ProProctor service who were disconnected that day, according to the Medical Council of Canada (MCC).

Tom Warren, vice-president of Prometric’s product line marketing, said the company is aware there was a technical systems issue that affected a number of candidates on June 12. The issue has since been resolved, he said in a statement to HuffPost.

“As with any new technology solution, we continue to evolve and work to yield a positive user experience for test takers,” Warren said. “We are committed to responding quickly and with urgency when unforeseen issues arise.”

Graduating medical school students are required to take part one of the Medical Council of Canada Qualifying Exam (MCCQE) at some point during their first year of residency. Although passing the exam is technically not a requirement to start residency, most students choose to get the test out of the way before their residency starts so they’re not busy with both on-call shifts and studying. The exam costs about $1,300, and most students register and pay in the previous fall.

Because of the COVID-19 pandemic, students had the option to take online exams, which are offered in Canada through Prometric. Some students have opted to wait to take the exam in person at a future date. But nine students who took the exam online reached out to HuffPost to share stories of spending days on their laptop booking the exam, being disconnected frequently and having to retake the exam.

While many students were disconnected June 12, the day Barsanti-Innes attempted the exam for the first time, students have experienced issues on other days, too. Dr. Rishi Sharma, director of education ofthe Canadian Federation of Medical Students, told HuffPost the organization has heard over 180 student complaints. One student reported that the exam took 16 hours in total with the disconnections.

In emails and interviews with HuffPost, students have detailed difficulties booking their online exams, frequent disconnections on the day of their exam, no proctor seemingly present and conducting their own security check to show they weren’t cheating.

Prometric’s promotional video outlines what to expect on test day:

Many students said they rushed through the exam, not reading over their work, so they could submit it out of fear they would be disconnected. At a time when medical students are already preparing for the start of their residency and looking to contribute to front-line pandemic efforts, many said the exam challenges caused additional stress to their mental health. Many students also said they feel the grades will be unfair because everyone had such a different exam experience.

The MCC will conduct a comprehensive review of this exam season, including evaluating the use of remote proctors, the council’s CEO said in a statement to HuffPost.

Dr. Maureen Topps said about 1,600 students of the approximately 7,000 expected to take the exam this year have completed it so far. It was moved online with the “best of intentions” so students didn’t experience a disruption to their residency, she said.

“However, it is clear based on the incidents that have been reported that for many the experience and, in some cases the outcome, has not been what we intended for candidates or expected and we are very sorry,” she said.

The MCC is working with senior leadership and technical teams at Prometric to resolve the issues, Topps said. The current exam period runs until midway through September. Students may be able to receive a provisional license through their province’s medical regulatory authority, per the MCC’s website.

Warren, from Prometric, said the company is dedicated to finding assessment solutions for every student. The number of candidates affected by the pandemic has been “unprecedented,” he said — the number of cancelled and rescheduled test appointments in the first two months of the pandemic has exceeded more than six times the company’s annual volume in the past two years.

Because of this, phone wait times have been longer, he said. Some call centres were also closed because of state and local guidelines.

Warren said the adoption of the online assessment for the MCCQE part one exam was historic, being the first all-day health-care licensing exam delivered through Prometric’s systems. He said 84 per cent of the candidates who took the exam from June 1 to 21 — almost 1,100 students — completed the exam with one or fewer restarts. “This is not a frequent issue,” he said.

Prometric is also reinstituting in-person testing at many centres as some restrictions lift, including in Canada, Warren said.

WATCH: Canadian medical students lead community drive to 3D print PPE. Story continues below.

College ‘failed to rise to these challenges’

Barsanti-Innes was able to reschedule her exam and write it without major issues, but it was two days after she moved cities for her residency. The situation has forced some students either to plan time out of their residency — where they’re supposed to be learning and developing patient-care skills — or to squeeze the exam in before.

“It’s really frustrating for a lot of us,” she said.

Dr. Wryan Helmeczi, a McMaster University graduate, in an email to the MCC shared with HuffPost, said he wrote the exam in seven hours with no food, water or bathroom breaks. He didn’t want a proctor to be greeted by an empty screen if he left while his app was disconnected and then it reconnected while he was gone.

Four of those hours were spent attempting to log on, showing his room, showing his clothing to prove he wasn’t cheating and waiting to speak to a proctor. After being disconnected and trying to get through to a phone line for help, he said he was finally able to finish the test at midnight.

“This treatment is unethical and inexcusable,” he wrote, adding that the challenges faced by students mean the exams can’t accurately reflect their abilities or knowledge.

“I understand these are difficult times, but I am at a loss for words for how profoundly the MCC has failed to rise to these challenges and their refusal to accept any responsibility,” Helmeczi said.

University of Toronto graduate Dr. Majd Abdullah, in an email sent to the MCC and shared with HuffPost, called the testing conditions “unprofessional and inhumane.” Abdullah said almost two hours into the exam, he needed to use the washroom, but didn’t receive any messages from the proctor after calling into the microphone.

He said he began experiencing abdominal discomfort and rushed through the remainder of the exam so he could use the washroom without worrying that his exam would be cancelled or disconnected in his absence.

Warren, from Prometric, said the company is aware that some candidates have reported concerns and issues about proctor responsiveness. “We reviewed these concerns and took appropriate action to ensure future compliance with established customer service standards,” he said. All Prometric proctors pass a certification exam that is renewed annually and have regular evaluations to ensure compliance with consistent processes and procedures, he said.

Prometric has an average ratio of eight students to one proctor with additional virtual support, he said.

Topps said students have a 45 minute lunch break and can take unscheduled breaks to access food, water or medication outside of the exam room.

Dr. Jenn She, a Queen’s University graduate, said another issue is that students who have been able to access accommodations for disabilities throughout their time in university had extra challenges booking the exam, because they had to do it over the phone and many calls went unanswered. One friend who had exam accommodations throughout university spent eight hours on the phone, trying to book his online exam, she said.

Experts say some tech issues inevitable for online exams

Darryl Hunter, an associate professor at the University of Alberta’s faculty of education, said for a high-stakes test with a credential riding on it, it’s incumbent on the agency administering the test to ensure nothing goes sideways.

But who is ultimately responsible in a case like this depends on the nature of the contract between students and the MCC, he said. If that contract said the student will be able to have the test be accessible and the MCC is unable to provide this, then the contract has been breached. But if it said barring a natural disaster or act of God, the MCC will try their best to make the test available, then the terms written into the contract allow for something like the pandemic, giving them an out.

A spokesperson for the MCC declined to provide a copy of the exam terms and conditions to HuffPost, saying they are a contractual document. The spokesperson did not answer HuffPost’s question about the wording of the contract, but said “the terms outline what is expected of candidates taking the exam.”

The overarching question of who bears the responsibility of test outcomes will be a question asked here, Hunter said. Without diminishing the negative experience of students, this could potentially be viewed as a small issue given the overall number writing the exam, he said. Hiccups and glitches can be expected as we continue digitizing education, he added.

Stella Lee, the director of Paradox Learning Inc. who works primarily in e-learning consulting, said there’s always an adjustment period when adapting to a new technology.

Even with online exams in more normal times done in a classroom, there can be tech issues, she noted. She said the length of this particular exam could make it a challenge for Prometric’s software, but it’s hard to tell whether issues stemmed from students’ own devices or internet connections, or the software itself.

Both experts said a rehearsal or trial test should be used in cases like this.

Warren, from Prometric, said the company has defined software development and deployment processes that include case testing and other quality control measures. It is continuing to evaluate and perform tests, he said. “There has been no deviation from these processes as it relates to support for the MCCQE Part I program.”

Lee said tech companies are responsible for doing stress tests, especially for testing that’s going to be scaled up. Educational institutions, or the institution administering the test, also have to take responsibility if anything goes wrong, she said.

“I understand this is uncharted territory for a lot of organizations, but I think they should take some responsibility to say, if technology fails — and it will fail — what next? What can you do to make sure the students are not being disadvantaged?”

Other remote proctor companies have also experienced technical issues before the pandemic, she said.