There have been a few recent blog posts after VisWeek about paper rejections such as Niklas’ Elmqvist’s Dealing with Rejection and Robert Kosara’s related Dealing with Paper Rejections. Rejections are certainly a painful part of the academic review process I’ve had my share of rejections.
A paper review is a criticism of a work, but, it’s not a dialogue. One of the painful aspects of a paper rejection is that you don’t get to address your reviewer: perhaps they misunderstood some aspect of the work, missed a key point. Or maybe they have some valid criticisms about some aspects of your work, but you can’t get more details from them which could be really useful in improving your work. Or, they may have some uncovered some other relevant confounding factors that you hadn’t addressed. Or, identified some other relevant prior work. In all the above scenarios you’re stuck without being able to engage your reviewer with more dialogue – and the review process is not the place for discussion as pointed out by Elmqvist and Kosara.
Critiques are very similar to criticism in paper reviews – however critiques are about back-and-forth dialogue, ideally in a face-to-face setting. Critique originates in 18th century Enlightenment when scholars and bourgeoisie were struggling against the absolutists in church and state. Critique is a distinct public discourse based on rational judgement (Eagleton). It is a public exchange of opinion, open to debate, attempts to convince, and invites contradiction (Hohendahl). This dialogue is really useful to explore, investigate, deliberate, explain, expand, probe, refine and revise ideas. The feedback is quick and many different aspects can be considered.
While conference papers are a good opportunity to get feedback after a paper has been written, a critique can be used to collect a lot of detailed feedback from a variety of peers and experts before a paper is written. This can help with authoring a better paper, and it can help guide better research by stimulating ideas, framing research, exposing assumptions and so on.
Furthermore, critiques from experts (peers, expert users, etc) represent a form of evaluation. Critiques are used often in design education (and medical education) where there is a lot of complexity and many tradeoff decisions. Unlike a traditional time and error test, a critique is wide-ranging across the broad design space and can uncover various unforeseen issues. Just like rejection, critiques can be difficult and painful for the person receiving the critique as issues are exposed. However, the dialogue allows the person to engage with the critic: to go deeper, to debate, to contradict, to understand, to accept, to learn. Note that formal critiques are the primary form of evaluation in design fields such as architecture.
Interestingly, some types of conferences are more open towards critical discussion than others. Marquee conferences (VisWeek, CHI, etc) are so big and diverse that it might be hard to generate much interest in a specific topic. The marquee conferences are so big that everyone is hurrying from session to session and you don’t necessarily get great group dialogue. Instead, many smaller workshops and smaller conferences are much better for engaging in dialogue directly related to a research topic. Acceptance rates tend to be higher at these more narrowly focused conferences. And, all attendees at these small workshops are focused on similar research and therefore more willing to engage in critical discussion. I.e. smaller conferences and workshops may be better than large conferences.
There are other ways of engaging in critiques as well, such as reaching out to experts via email, blogs, skype, doctoral colloquiums and other means. “Do-it-yourself critiques” and other aspects of critiques are discussed more in my recent paper at BELIV back in October at the BELIV workshop at VisWeek; or, for the abbreviated version, here are the slides.