A Question for My Fellow Panelists
By Ed FeltenTo motivate my question, let me tell an exaggerated story about how the Internet moves us from an Old System to a New System.
In the Old System, a scholar writes a paper, the paper is peer-reviewed and published in a journal, and only then is the paper read by the community.
In the New System, a scholar writes a paper, revises it based on feedback from a few colleagues, and then posts it on his web site. The community reads and discusses the paper, and only later — after most of the community has read the paper — does the paper undergo formal peer review and publication. In this New System, formalized peer review and publication does not serve the gatekeeping function of determining what the community reads, but instead provides a retroactive stamp of approval that typically reflects the community’s collective verdict on the paper. Formal peer review is augmented by informal discussion within the community, which starts as soon as a paper becomes available.
This story is an exaggeration, of course. We have been, and will continue to be, somewhere in the continuum between the Old and New Systems. But my sense, based on practices among computer science researchers, is that we have moved some distance toward the New System.
So let me ask my fellow panelists: Do you see the same trend in your community? Is this trend good? What else will the Net (and infotech generally) do to scholarly communication?
October 17th, 2007 at 1:08 pm
The advantage of the Old System is that the system of peer-review provides a gate-keeping service, as you rightly point out. The merits of this methodology have been debated, but it’s clear that (although there are disadvantages) the system works well enough to keep scholarship moving forward.
However, it’s difficult to say that we’re in a transition between “the Old System” and “the New System” because it’s not entirely clear what the characteristics of the New System are (or ought to be). I’ll agree that we know the “Old System” is on the way out, but where we’re going from here isn’t clear enough to really classify what scholarship is going to look like in a decade.
So the question is, what do we need out of a system of publication? We need some level of gate-keeping, to make sure that the peer review process is *starting* with material of some quality. We need some level of peer review, this is certain (if the Internet has shown nothing else, it’s shown that there is a staggeringly huge quantity of bad scholarship out there). We need a system of rewards that encourages good scholarship, and (equally so) we need some system of penalties that helps offset the incentives towards bad scholarship. We don’t have this now; science in particular has come under assault as a scholarly discipline by forces outside academia who pursue other agendas.
I think the best course for academic journals is to change themselves from a “collect, review, disseminate” model to a “scholarly gathering, enabled collaboration, publicize” model. The gate-keeping function needs to be preserved, but there are two stages of gate-keeping. The first is helping direct what the *community* is reading, and the second is directing what the community presents to the rest of the world as worthwhile material.
In more than one sense, scholarly journals already provide some level of community (look at the various IEEE journals and their respective areas of focus), but right now the real community function is performed “one step removed” from the journals - your academic community more or less says, “We all read these 3-5 journals, and most of us read some of these other 5-8 journals, and specialists in these areas read these other journals, too”.
The publication functionality needs to move “up” a step and integrate more with the communities themselves.
October 17th, 2007 at 1:39 pm
[…] New System supports two good things that facilitate research: immediate access and open […]
October 17th, 2007 at 6:19 pm
Wouldn’t it be appropriate to speculate about the “next generation” system… that is, the system after the “New System”. It would seem that things like “papers” might themselves become obsolete. We already see this in a few places where someone posts a preprint with a version or date number on it and may update it as often as daily… that makes set-in-stone papers seem sort of quaint. I must admit, having to work on the papers I’ve written well into the future seems like it could be very daunting… there’s some comfort in pushing something through a journal/conference process and letting things lie.
October 21st, 2007 at 12:05 pm
And just to complicate things further, let’s imagine how the New System will work (or not) for monograph publishing. Is it reasonable to have scholars to post preliminary versions of entire monographs and expect the “community” to spend its time collectively reviewing them all? We have a difficult enough time, as a university press, engaging just two scholars to review a monograph, even while offering a modest fee. Surely, the busiest and best scholars are not going to be allocating their precious time to providing thorough reviews of hundreds of monographs posted in their fields with no compensation at all! Also consider the irrationality now built into the current system, where dissertations are available electronically through ProQuest to which many libraries subscribe and where for that very reason libraries feel reluctant to purchase monographs based on dissertations, leading to the result that presses are loath to publish revised dissertations on which the whole promotion-and-tenure system depends for the career advancement of junior scholars. The New System for monographs would very likely lead to the same result: nonpurchase by libraries of any works posted electronically in earlier versions, hence decisions by presses not to publish them, hence failure to advance their careers by the authors of these works because they can’t get contracts from presses.