12/11/2011 12:50 EST | Updated 02/09/2012 05:12 EST

The Innovation Gap in Canadian Academic Research

At its best, peer review ensures published findings in academic papers are based on scholarship that is meaningful, relevant, and credible. At its worst, peer review is slow, expensive, and nowhere near impartial. The problem is that peer review has been shown to re-institute orthodoxy as new discoveries.

This part two of a three-part series co-authored by Johannes Wheeldon, PhD and Richard Gordon, PhD. The authors of the series have explored this issue from different angles elsewhere: Richard has demonstrated how the cost of rejecting a research grant now exceeds the cost of giving one for qualified Natural Sciences & Engineering Research Council Canada (NSERC) applicants; Johannes is currently involved in a case in the Federal Court of Canada, that will finally define what sort of feedback is required to failed doctoral and postdoctoral applicants.

Previously we argued that despite the importance of research to Canadians, Canada's existing research funding model creates motivation problems for researchers, discouraged by a federal funding funnel that rewards and therefore supports too few researchers. In this post, we discuss the consequences of the current grant funding model and the specific ways in which this model undermines innovation in Canada.

The peer review process, in theory, provides a means for quality control and self-regulation in academia. Peer review by scholarly peers and experts in a specific field can offer researchers independent and constructive criticism. At its best, peer review ensures published findings in academic papers are based on scholarship that is meaningful, relevant, and credible. At its worst, peer review is slow, expensive, and nowhere near impartial. Concerns about the ability and effectiveness of peer review to ensure quality control are growing. In the end, peer review is a human endeavor and thus subject to the regular foibles of people and their own personal politics.

While peer review remains important for evaluating younger scholars and for quality control in journals reporting research results, it has no place in the grant funding of new, untested ideas by scholars with demonstrable records of achievement. The problem is that peer review has been shown to re-institute orthodoxy as new discoveries, theories, ideas, or hypotheses must overcome the conservative propensities of the peer review process. Scholars have long pointed out:

(Peer review tends to) operate on the negative Achilles heel principle of killing any applicant with any, even minor, discernible weakness, rather than the positive principle of giving the applicant the benefit of the doubt -- since he/she is the most expert of all in his/her own particular field.

As anyone with any knowledge of science knows, it simply is not possible to predict which new ideas may lead to the discovery of penicillin, electricity, or a host of new treatments, processes, and products which many of us rely upon today. Yet the peer review system presumes it can. The problem is old:

Innovative work (because it is new and not yet understood by many people) is more likely to get bad reviews than routine research whose basis is widely accepted; thus there is a strong bias in the "selection" process against innovative research.

Relying on this system, as presently applied, also has administrative consequences that go beyond the enormous time it takes to prepare a grant application. Collaborators must write endless letters of support, committee members must be found, and mountains of paper must be reviewed. Reports from the front lines describe a process that is full of vagaries, inconsistencies, and bias. While some perhaps more cynical scholars might suggest the grant process at present is designed to keep academics in line, tethered to their computers and departments, it is more likely that administrative inertia and aggrandizement has set in. Either way, at present, researchers are being asked to spend too much time begging for money and not enough time doing research. Research productivity could rise significantly if the requirements to write grant applications were reformed. This could in turn renew Canadian excellence and innovation and allow scholars to focus on research, not grantsmanship.

Perhaps the most pressing issue from a research integrity point of view is that the present system seems to deny that doubt permeates all science. Granting councils appear to believe the peer review process cannot ever be wrong, nor even briefly mistaken. Appeals are few and nearly impossible to win. Whatever the good intentions of those who serve the existing model, Canadian taxpayers currently fund competitions through the Social Sciences and Humanities Research Council Canada (SSHRC), for example, that provide no explanation for the scores on an application and no justification for a rejection that can significantly harm an applicant's academic career. A system based on denying basic accountability to applicants, taxpayers, or the federal court itself cannot foster innovation or promote transparent scholarly research.

At issue is how this arrogance feeds into the widespread war on science, logic, and reason. There is a growing sense that scientists too often focus on profitable book and industry deals and television shows over the slow and less glamorous accretion of empirical evidence and explanations that help society better understand the world. It is no wonder people are skeptical when those who profess expertise fail to explicitly acknowledge limitations that are part of all research. In Canada, the Tri-Councils (SSHRC, NSERC and CIHR) are feeding the myth of the academic superstar, instead of funding diverse research. The result has led to an "internal Canadian brain drain" in which:

... disenfranchised researchers [are] lost to the research enterprise and continue to be paid by their universities for work the present system prevents them from doing -- a two-fold waste of taxpayers' dollars... thousands of good heads across Canada who cannot deliver the goods.

So, what can be done? In the final part of this series we will turn our attention to some reform options.