I believe peer review should be supplemented or even replaced by social review methods where not only arbitrary reviewers but the whole scientific community might have chance judging, discussing and commenting any paper. Online.
The logic and safeguards may not be that easy to create in the first place but in my opinion it would worth the effort eventually!
We will never be all experts on all subjects. Peer review is by peers, not laypersons.
From the STEM perspective, a democratic solution would be a disaster. Two immediate reasons come to mind, the loss of peer expertise in the noise, and brigading.
I wrote "scientific community".
What I meant is the "relevant scientific community", it wasn't evident apparently.
Also the selection of the reviewers is just partially depends on the expertness already, several other aspects affect it quite a lot. Not to mention that why a certain selection should be the one why not an other, why not the relevant community chooses the reviewers then?
Just because not every details are fined carved the idea should not be dropped.
(I was participating in certain peer review processes where I was an almost outsider and very far from being an expert, I have little conviction that the current one works well)
Imagine that you have voting rights to review a paper (a-la slashdot, where random people got opportunity to tag something as insightful, interesting, etc).
Now imagine that there comes an article in X subject (say, Agent Based Modeling).
When you "vote" in that article, the "dimension" of your vote is proportional to your "impact factor" in that subject (i.e., say you published 20 articles in ABM and you got 10 "votes" on them, then each of your votes count as 10 votes). On the contrary, if your impact factor is negative, your vote doesn't count. That way people that are considered "knowledgable" in their subject, will be able to peer-review other articles.
Another method would be something like what StackOverflow has: Initially everybody gets 1 vote (or 10, or 1 every month, or whatever), and you "transfer it" by voting for an article (maybe to the 1st author, or evenly distributed), so because the "votes" are scarce, people with care for them. And people with articles that are most voted, can themselves vote more.
There are plenty of systems that could work. And the beauty of it is that they could be "layered" on top of Arxiv with a Chrome extension or similar.
There seem to be more effective ones and less effective ones.
Unfortunately sometimes economic incentives cause them to be gamed against real quality. E.g. think of shill online shopping reviews, and circular voting to boost reputation.
In academic terms, that would be "citation rings" to promote their rankings. Like web rings, there are nice and friendly ones, and there are heavy, spammy clones of sites. I would expect the rise of junk-article, plagiarism-from-elsewhere, machine-learning-assisted-plausibility mutual citation rings if there were no good controls to detect and prevent that sort of thing.
Hm, perhaps the HN should be closed as well and everyone should have their own blog instead?... or ask the peer reviewers to discuss and judge papers on their blogs instead?
But you are aware of journal articles as well, dont't you?
And how standardized and comprehensive the peer selection is?
(it is not, in general, with huge variations)
> But you are aware of journal articles as well, dont't you?
Yes.
> And how standardized and comprehensive the peer selection is? (it is not, in general, with huge variations)
Yes.
So what?
My point was you can accept whatever papers you want. You can't make or stop someone else accepting. Seems like a fine situation to me? Anyone can recognise the papers they want to.
