Why don’t research groups just publish to their own websites or directories like arxiv? What’s the role of an academic journal in 2020?
Honest question, I’d love to see more blogging from hard science academics but I’m wondering if there’s a reason why that’s challenging or if it’s just academic culture. We should have a Substack/OnlyFans for scientists.
1. Plenty of scientists do. In biology, medRxiv and bioRxiv are pretty popular. For example, all publications from my lab are first put there (www.kosurilab.org/publications.html). If we have smaller pieces of work, we tend to just open source them. It's not widespread practice, but it's definitely not uncommon.
2. Plenty of good blogs, open source results & protocols, a strong github community in academia. Again, not widespread, but not uncommon either.
3. The role of academic journal is attention. There are heirachy of journals meant to signal quality, and they are hard to get into. They are very very useful as a signal for future career prospects (much in the same way as going to a good school).
Speaking from my own experience in the physical sciences, labs don't self publish unsurprising results because there are only so many hours in the day and it's not worth the effort.
Even just putting the results on your own website is a lot of work. Pulling all of the data together, analyzing it, putting it in visuals, writing up the results. It can be hard to justify committing that much time to something where the pay off is "other people might be interested".
Hmm... it seems like this would be a good thing for undergrads to do in a class setting or an internship. It would give them experience writing an actual paper, albeit with null results.
It would require a lot of babysitting. Some objects have a few slightly different definitions (for example,one book has a definition, and the other book the definition is 1/2 of that). Sometimes the programs to run the calculations don't use the same variables than the paper (perhaps the team changed the opinion, or the main reference, and the graph must show x+y vs y). Sometimes the work that should be included in the paper is underdocumented, ...
Another difficult part is to select what to publish, for example cut the dead branches and add a few more data about the interesting part. It is not usual to get a bunch of data and just publish it without some additional work.
Yep, and it’s not really how authorship is supposed to work.
The hard part here is that communicating any idea clearly to an audience takes massive effort, and usually null results, unless quite interesting in a specific context, are naturally a lower priority.
I’m currently working on a paper built around what I believe to be a fascinating null result though...
I'm not sure what that means, but we are not using it.
In medicine some studies are preregistered, but one of the lessons of Covid-19 is that each week there is a new study that is clearly unregistered, without a control group or with a definition of control group that makes me cry (like "an unrelated bunch of guys in another city").
I think the people in particle physics have a clear process to "register" what they are going to measure and exactly how they are going to processes it. (The measurements are too expensive and too noisy, so it is very easy to cheat involuntarily if you don't have a clear predefined method.) Anyway, I don't expect them to have the paper prewritten with a placeholder for {hint, evidence, discovery}.
In most areas you just put in the blender whatever your hearth says and hope the best. Or run a custom 5K LOC Fortran 77 program (Fortran 90 is for hipsters).
If you get an interesting result for X+A, Y+A and Y+B, you probably try X+B before publishing because the referee may ask, or more B because B looks promising.
If you run a simulation for N=10, 20, 30 and get something interesting, you try to run it for N=40 and N=50 if the interesting part is when N is big, or for N=15 and N=25 if the program is too slow and the range is interesting enough.
And it is even more difficult in math. You can't preregister something like "... and in page 25 if we are desperate we will try integration by parts ...".
That, and on the other side of the equation, there’s not enough time to read the stack of papers I already know to be deeply interesting. I could read in my subfield of neuroscience 24/7 and never catch up with the deluge of new, interesting, and high quality work. I agree that negative results should be publishable, but the whole incentive system in science must change to accommodate that.
Copyright issues too. A lot of journals require that they have the sole copyright to the work they publish. If you have already published a portion of the work on your blog or whatever, then things can get lawyer-y.
So the same reason programmers don't document our code? We don't realize that without communicating our work, others can't make use of it. I think this is even more true in science than in programming.
For me, trustworthiness. Not all journals are equal but some are held in such high esteem that I would grant a lot more credence to the findings of an article published in a journal than one self-published.
I have neither the time or will to assess the merits of each individual who might publish something. If a name is sufficiently big within a field, it's usually in a sufficiently big journal; on the other hand, I would treat a self-published article with the same level of skepticism as those low-tier journals that happily publish pseudoscience as fact.
Call it snobbishness if you will, and I'm fully aware that academia is full of it, but that's the role that a journal fills for me.
If it's good research, it's more likely to be published in an academic venue (and possibly the other way around). So if there are a lot of papers, I will prefer the academically published ones.
I have limited time to read publications, I am not going to read everything just because it is available somewhere. For me, the amount of papers published these days is an argument for academic venues, not against them.
(As I am a computer scientist I have a hard time writing "journals" instead of any more general term, as conferences are way too popular in this field.)
One cool thing about "unsurprising" results—trustworthiness of the source doesn't mean much more than fitting your preconceptions in any way that reviewers can tease out anyway.
If a self-published article included full equipment and software to replicate their results in a multimedia fashion (basically included engineering), would that alter your snob-ness?
i think it's largely culture. academics are "graded" based on how many papers they publish. depending on your geography, quality matters too.
but in most cases they aren't measured by how much non-peer reviewed publications they have.
there is also a technical barrier that works both ways. most don't have the capability to have a regularly updated website with content that otherwise would have been put in the file drawer. and the other side of that coin is what audience will actually find / read it online.
twitter is becoming the defacto medium of dissemination, however, so that may bode well for promoting other types of publishing medium.
edit
ALSO note that there are some academics who publish high-quality blogs. i'm thinking of [murat demirbas](http://muratbuffalo.blogspot.com/). i'm not familiar with any that publish actual results, however. the threat would be someone else might "steal" the idea and publish it elsewhere, particularly in competitive fields like bio.
> there is also a technical barrier that works both ways. most don't have the capability to have a regularly updated website with content that otherwise would have been put in the file drawer. and the other side of that coin is what audience will actually find / read it online.
Add to this: the ability to get things indexed properly in google (scholar), easy way to update metadata, server availability, dois, apis, etc. I contract for neliti.com and we provide stuff like this now for orgs/journals and conferences all over the world outside of the US for en, id, tr, ru, uk, es, pt, and ms locale content (pdfs,xmls,docx, datatests, etc.) but mostly a lot of indoensian orgs/journals and conferences, with ~40 using their own custom (sub) domain we route for.
I hope that eventually the landscape moves to a place where these services can be provided for individuals too (lots of upstream stuff like dois requires having an organization which i think is stupid in this day in age where any piece of digital content can easily get an identifier, related org or not).
Status. Brand. Trust. That's all it's really about. If Newton and Einstein started a curated website of the best papers of the year, those papers would probably experience increased citations, and you would probably pay a premium to subscribe to that list.
Well, as for non-‘vanity journals’, I would say the anonymous peer review process. Similarly, the publishing process after the initial anon peer review process can be generative + a healthy back and forth revision process. Plus it has the potential to take you out of sub-sub discipline echo chambers w/r/t some aspects of methodologies and other approaches to even writing/explication. Largely, though, I would say from both sides, (I.e. reader and scholar), the journal and all that comes with it (I.e. peer review process, the academic press and their branding, in some disciplines the main editors can also lend something to this) lend a greater degree of credibility and confidence in the expertise of the scholar(s). I mean this in a sort of take it for what it is, the positive and the unseemly all at once, sort of way.
There are sociology studies, published in closed journals, that detail the positive impacts that public libraries have on communities. The PhDs of that field must either be (1) blind to the irony, (2) totally resigned to this state of affairs, or (3) themselves complicit as managers of closed journals.
One more indictment: I'll bet they're all extremely depressed about how scientifically illiterate the general population is. They've probably published papers about that too, which the public can never read.
Some people see the devil in the Koch brothers, some people see it in Trump. I see it in the managers of closed journals. If you possess scientific knowledge and hoard it away behind a paywall, you are going to science hell. If you make a scientific discovery and sign away the publishing rights to somebody who intends to hoard it in this way, then you are spending at least a few years in science purgatory.
Well said. I don’t understand why public institutions at the very least aren’t required to share their research. And considering that the majority of higher ed public or private wouldn’t last a day without federal backed loans, I’m not sure only public institutions should be held to this fire.
I take actual pleasure in pirating research papers.
(I also post the PDFs and links to code on my website, as well.)
> What’s the role of an academic journal in 2020?
Partly peer review, partly signaling game, and partly exposition/advertisement/reach of papers. Those are, of course, all intimately linked in scientific fields.
> I’d love to see more blogging from hard science academics
I think a lot of them have taken the Twitter route! There are a few who also have their own somewhat-updated blogs (myself included).
> I’m wondering if there’s a reason why that’s challenging or if it’s just academic culture.
The problem is kind of general, though: (a) writing good, useful blog posts takes a long time, especially when attempting to distill the topic even further for a general audience, and (b) writing (good) papers really does take a long time. (Of course, there are many, many poorly-written papers, but, unless the results are truly incredible, almost nobody spends time deciphering them.)
For example, I think I'm a fairly fast/decent academic writer, yet on a given paper, I spend roughly half of the time doing research and the other half of the time finding the right presentation/abstraction to present, along with writing and editing a given exposition to make it clear and legible. Any given paper will take me ≥20 total hours in structuring, writing, and editing (not including research hours). Reviews can take ≥40.
I have very limited research experience (2 undergrad practicums) but the professor I worked with was a leader in their field and sat on a journal review panel along with others of similar standing. So I suppose consensus among colleagues is " x,y and z are leaders on a topic, their opinions are in b journal so I should read that."
One reason is prestige. This is not a fluffy concept in academia - publishing in prestigious journals means funding and tenure for professors and good job prospects for grad students.
Another is time. Every minute spent blogging is a minute spent not doing the above.
Another is curation. Frankly, Nature, Cell, Science feature work from the biggest, most well-funded labs. They have a lot of stuff masquerading as novel, but they also feature pioneering, transformative work. Science is about methods, and the fancy journals are where you see methods first.
Its a tragedy of the commons situation, sort of. We are all very aware of the problems with academic publishing.
(At least it used to be because) the paper's publication score, which is relevant for your quantified objectives (universities run by administrators and all), is weighted by the journal's impact score, so e.g. one paper in Nature is 'worth' more than 10 in conference proceedings.
Most people do publish 'working drafts', in practice the same paper submitted to the journal, on their own sites and to archives.
We should have less transparency, not more. Right now the peer review process is single-blind, but it really should be double-blind.
Even if someone doesn't have a PhD or work at a research institution, they should be able to publish good science. Right now, that just isn't possible. And the opposite problem is also true: if you're a big shot in your field, you'll be able to find at least one journal that will publish whatever crap you submitted, regardless of the quality.
Getting that benefit only requires double-blinding during the review process. There’s no reason that both sides of the blinding cannot be removed (and also revealed to third parties) after the review process is complete.
I can't think of any real upside of that, but can think of a lot of downside. Humans, and especially academics (I've noticed a trend in my work environments: the less money everyone makes, the more power is sublimated into inane dick measuring contests) are petty, and less personal it is, the better.
2. The reviewers are each assigned a global permanent pseudonymous identifier—a UUID, basically—known only to them and some "Society for the Advancement of the Scientific Process in Academia" organization.
3. Every vote a peer-reviewer makes, and also every opinion they write about a paper, must be registered with the same academic-process org, whose job is then to collate and publish them to the Internet under the reviewer's pseudonymous identifier.
You'd be able to use such a website to both 1. audit the peer-review process for a given paper; and 2. cross-reference a given peer-reviewer's votes/opinions.
Additionally, the standards body itself could use the cross-referencing ability to normalize peer-reviewer votes, ala how the Netflix Prize recommendation systems normalized votes by a person's interpretation of the star ratings. (They'd have to ask peer-reviewers to vote with something more fine-grained than a binary pass/fail, but that'd be an easy change.)
The only thing I would worry about in such a system, is that academics might not want the negative opinions of the peer-reviewers on their paper to pop up when random other people plug the paper's DOI into Google Scholar, because a dissent on an accepted paper might unduly impact the paper's impact-factor.
You can't think of any reason? What about retaliation for being the less enthusiastic reviewer? What about the opposite, that you become known for being an easy reviewer who missed some obvious flaws?
You're still assuming that this happens right away. How about if the blinding is removed after 50 years? Then people studying the history of science would have the data, but it would have no impact on the careers of the people involved.
Indeed. I've seen some mightily acerbic rebuttals to other researchers' articles published as articles; I dread to think what a rebuttal to a review might look like.
I'm sure someone will say "well, maybe the acerbicness is the problem"; perhaps so, but I welcome the rigorous honesty with which some academics willingly write.
Institutional bureaucracy only cares about official journals. People in real fields use arXiv anyway - the more the researchers use arXiv or similar, the realer the field.
Luckily not everyone feels this way, and may be willing to mail an author if they found some mistake in their open archived paper/data/document they read because they think it could be useful/interesting and the author may be willing to update it with all the relevant metadata acknowledging the edit.
The more of academia that happens outside of well defined institutions, the better for the public.
but journals and peer-review enforce this. That's a good thing (though I agree that letting journals do this is stupid – arxiv is out there, why can't they sign up unis to make people review the things there.)
if you do true double-blind reviews and don't do the thing, they do at the great CS conferences (e.g. chair is from FANG, 80% of papers are from FANG; chair passes everything from his FANG).
Open science is great, doing the mixture of boasting over mediocre results (hey, we can classify 10 samples more correctly from MNIST) and outright faking stuff (which is also there) is a problem even with journals. It doesn't get better wirh Arxiv and all the details packed away in never published supporting information (happened to me sometimes now...) - of course you can ask and then with the SI realize: this is bullshit... Good peer review should weed this out.
I don't deny that "good" (people may have varying definitions of good, acceptable costs of obtaining sufficient good-ness, whether those costs are assumed to be burdened upon a individual or society unquestioningly, on whether such costs can be minimized and how such minimization might negatively affect those not directly integral to academic pursuits and more integral a current system of manifesting such at non minimized costs) peer review may weed these behaviors out more often than not (boasting over mediocre results, outright faking stuff, etc), but for me, I'm not too concerned because this stuff will happen no matter what to some degree (like you say `It doesn't get better wirh Arxiv`). We are only human.
I'm more interested in making the _costs_ of publishing/disseminating/accessing research cheaper so that individuals/orgs/etc can bypass expensive institutions if they so choose, or even gain the ability to participate because they have no other realistic options now due to current costs of participation in the ecosystem (I personally have to deal with thousands of journals/conferences/orgs every week that use software I write at significantly cheaper costs than say systems MIT has at its disposal, that without such, their works would not be available to society [i've see domains i route for go down when they stopped paying their bills for a month or longer, and we had to redirect publications/data back out from domain so that it could still be available]).
Any system that can drive down the costs for pursing academic interests for both the individual and society will eventually subvert those that came before, relegating them a hollow shell of the purpose they once served, if any aspect of a previous system manages to survive.
Honest question, I’d love to see more blogging from hard science academics but I’m wondering if there’s a reason why that’s challenging or if it’s just academic culture. We should have a Substack/OnlyFans for scientists.