A favorite conversation topic at computer science conferences is griping about computer science conferences. While we may not agree on the specifics, everyone seems to concur that the list of shortcomings is long.
Popular complaints range from the ambitiously idealistic to the mundanely bureaucratic:
- Nondeterministic reviewer assignment makes authors play Whac-A-Mole with wildly shifting criticisms from submission to resubmission.
- Secret reviews and high per-PC-member load incentivize shoddy reviewing.
- The latency from submission to presentation—typically six months or more—is too long.
- Once-a-year conference deadlines incur undue tolls on grad-student stress levels compared to monthly review cycles.
- Overhead from ACM or IEEE affiliation precludes reasonably priced open-access publication. (“How much can it possibly cost to serve a bunch of PDFs?”)
These issues, in some combination, have appeared at every conference business meeting I’ve attended. Unsurprisingly, the system is slow to change. Innovation is risky and requires time investment; steering committees are understandably risk-averse and populated by busy senior faculty. Some prominent people on the publishing side don’t believe there is a problem.
We should keep pushing for incremental change within the current system. At the same time, there’s room for a different approach. We can attack the stagnation from the outside in: we can start something new. The new thing will need to start from scratch with earning attention and credibility, but it could break with tradition more rapidly.
We should take inspiration from other communities that have been more proactive in adopting new publication norms: the arXiv for unreviewed preprints, open-access journals like the Forum of Mathematics, and—closer to home—VLDB, a notable computer science exception to conference conservatism.
A Lightweight, Independent, Open-Access Journal
It’s time for new, independent journals. A Web-only journal has low overhead, in both time and money, and can work as an experiment for new ideas in publishing.
If I were to start a new journal today, it would look like this:
- Fast reviewing cycle: a monthly or rolling deadline, with a decision promised in one month. Decisions are four-valued: reject, major revisions, minor revisions, accept. Major revisions means you can resubmit and we’ll make an effort to assign you the same reviewers who recommended the revisions.
- Broad reviewing base: as with a conference’s external review committee, the editors select the best matches from a long list of volunteers who expect to be contacted only occasionally.
- No meddling: unlike most current journals, we do not copy-edit or meticulously format. We post PDFs exactly as the authors upload them. Instead, the journal could accompany papers with their reviews, an editorial introduction, or moderated public comments.
- “Diamond” open access: free to publish, free to read. Web hosting is our only real cost; this is minuscule, easily covered by a sponsor university or grants.
The goal of these design decisions is to keep the notional new journal lightweight: to avoid the conservatism and overheads intrinsic to established venues. Aside from these basic tenets, a lightweight journal could also conduct other experiments: for example, publishing source code or data along with every paper.
Next Steps
Starting a new journal poses clear risks. It will need to address the chicken-and-egg problem of credibility: authors will submit and reviewers will volunteer only if the venue seems worthwhile. The time commitment will initially be dubious for such an untested idea.
But one challenge I don’t worry about is infrastructure. The tools we need to build a Web-based review and dissemination system are free, high quality, and open source. The baggage of Manuscript Central is eminently avoidable and reliable hosting is cheaper than ever.
The project is risky, but as researchers we should be familiar with long bets.
I want to hear from other computer scientists: What are your favorite complaints about the conference system? What could we build that would sidestep them? How can a new, lightweight publication venue achieve relevance? I know you have great ideas—please get in touch.
My thanks to Brandon Holt, Amrita Mazumdar, Pavel Panchekha, and John Toman for reading drafts of this post.