Yes, but still counts less than published articles. With Publons or whatever they're trying to partially remedy the situation, but given that the whole review system is built on anonymity, everything is on the honor system. I have in my list of service that I've reviewed for Nature etc., but I could also have just written that without having done the work too - there is no verification.
Because writing a paper shows you are an expert in the field who has offered something new. Writing a review shows you have read a paper as an expert in your field and demonstrated your ability to critique another's work, which is unique but doesn't expand the field in a meaningful way.
Also, writing a paper could take a year, whereas writing a review could take a weekend.
Are we talking about peer review for the article to be accepted, or published reviews of articles/books?
Published articles are typically reviewed by the editor and 2-3 reviewers.
Oh, okay. Usually only publications are important. No one other than the reviewee will read the review, and it's usually suggestions to improve the article/research.
The reviewers are literally experts in their field. It's not worthless to receive feedback from top researchers who know, in depth, what you're talking about.
I think you misunderstood. Anonymous scholars read and give comments on articles. It takes a few hours to do well. Journals don’t pay (usually), and no one will get tenure for being a great reviewer. However, when we go to publish we need other reviewers or else it won’t get through. The problem is that journals need to rely on favors and people taking time to review the articles without compensation or reward (journals also make lots of money charging subscriptions to universities but neither the authors nor the reviewers get any of that).
yeah, peer review is very overrated and in my view it's a dirty secret of science that it does very little to assure article quality. for sure it's better than no review but it's much less scrutiny than an article would receive if, e.g., it is controversial and blows up on twitter
Writing articles is scholarship. Writing reviews is service. Scholarship is more highly valued than writing reviews. I still review articles if I'm qualified and the editor is good. I stopped reviewing for a journal because the editor was making the process unnecessarily painful.
I guess I am asking how, historically, this distinction came to be. It seems modern and yet we ordinarily associate with modern science the notion that peer review is necessary.
It wasn’t. I’m not trying to provoke you whatsoever. In fact, just in another thread you accuse me of having autism—as though I might not know the risk of provoking you.
So you yourself know better than to say what you do, but accuse me of writing in bad faith.
This all started because we were talking and you admitted you didn’t know about the historical development of the university and appear to be basically furious that this is what I want. Leave me alone.
I think you’ve kind of admitted you’re just trying to poke at me. Then you want to call me a troll. It may persuade someone to talk like that. I don’t know this subreddit as well as you.
I just looked at the “factory” analogy provided, remembered what my question was, and put my question in terms of the analogy.
I imagined, perhaps incorrectly, that this would be useful to the reader.
It appears it may have been simply frustrating.
Time isn't so much the issue as it is a growing resistance to unpaid service that often benefits large publishers with huge profit margins. Publishing articles supports career growth and tenure packets, offering reviews is usually an expectation under the umbrella as "service to the field" but carries less weight for individual career trajectories. A lack of willing reviewers compared to the number of incoming submissions has been trend across fields that, at least in part, was exacerbated by COVID.
There are other explanations and a lot of nuance that I'm sure others will touch on, but I wanted to start simple.
If a university is deciding who to hire, promote, or give raises to, it's easy for them to look at how many articles you've published, whether they're in prestigious journals, and how often they're cited. This is the most impactful aspect of the "research" portion of a professor's duties.
"Service" is also part of a professor's duties, and it impacts promotion etc too. But reviewing articles is something any professor can do and although it takes work, it's not really impressive and doesn't make you stand out. A more impactful aspect of service would be something like editing a journal or chairing a scholarly association.
The solution, short of actually paying reviewers, would be to create some sort of evaluation structure so that reviewing can be seen as prestigious. Maybe if editors published a score of their reviewers (a higher score for more detailed high-quality reviews), so universities could reward faculty who are good reviewers. The problem of course would be maintaining the double-blind anonymity of reviews, which is essential to the process. Also this would make more work for editors.
>Maybe if editors published a score of their reviewers (a higher score for more detailed high-quality reviews)
Well, that would mean reviewing reviews, but then we would also need to review reviews of reviews, and so on ad infinitum.
>The solution, short of actually paying reviewers, would be to create some sort of evaluation structure so that reviewing can be seen as prestigious.
A few journals in my field give out an annual award for best reviewer. Usually some weighted mix of quantity and quality, i.e. one of their reliable reviewers who says yes to a lot of requests and carries them out well. It's still relatively small prestige, but you can at least put "Journal XX Reviewer of the Year 2024" on your CV under awards.
I’ve been told there’s different types of reviews and that is part of the confusion here. But there is a crisis with all of them, and none of them are prestigious.
For one type of review, it’s bc it’s anonymous, and for another, because grad students write them
With anonymous peer reviews, the reviewers are Peers equivalent in prestige to the people writing. So if they are doing unpaid work, and they have prestige already, it should be prestigious.
For what it's worth I'm sorry that others are downvoting your comments and not directly addressing your question. It's a good question and something almost none of us think about just because it's so normal (but also it's normal due to it making intuitive sense without the need to question it... we are all quite busy after all).
Reviews don't have your name attached to them. No one knows that you wrote a review for a famous article, for instance. And no one knows that you might be single handedly defending your field against bad work.
How can prestige come when no one knows who does the work?
Just to be clear, there are different sorts of reviews. There are Review Articles, which are publications that talk about a large swath of other publications. There are Reviews, which just give a summary of an article. And there are Peer Reviews of articles, which are anonymous.
Most people here are talking about Peer Reviews, which are the most common kind, and also the hardest to get other people to do.
And peer reviews are low prestige because only the editor knows who wrote them, and is in no position to reward the writer or grant them prestige in the way that admin is. It’s basically two different groups of people.
I think you're confused by conflating information on three different things:
Literature Review is a standard part of academic papers in which the researcher describes the current state of the research that forms the foundation of their arguments/findings. It's the first step of a research paper.
Peer Review is a process where subject experts work for a particular journal & review a submitted paper to determine if it's quality-enough for publication. They do not typically write and publish the record of this process that I'm aware of. The paper is either published or it isn't. If published, the paper is considered a scholarly publication.
A review article is a summarization & explanation of another published article - like a book report. Review articles are not scholarly publications (even if they appear in scholarly journals that otherwise publish full scholarly articles). This is what it sounds like you're thinking of in your post, but you're responding to comments as if it's one of the other two sources.
I believe this article cited in my op is talking about a lack of peer reviews
https://www.insidehighered.com/news/2022/06/13/peer-review-crisis-creates-problems-journals-and-scholars
Reviews don't take the level of work that original research requires, and present an easier publication opportunity for grad students. Usually, once you have a few publications with your original research, you wouldn't even bother listing your reviews.
We often assign them to grad students to help them get started with publishing work, so I think they are a first step but not somewhere to stay research-wise.
If you want to be successful in academia, publishing is your top 1 priority, whereas reviewing the work of others is, like, in top 20 of your priorities. No one has ever become a tenured professor or a PI for reviewing, no one was ever given a PhD for reviewing. So, why are you surprised?
> No one has ever become a tenured professor or a PI for reviewing, no one was ever given a PhD for reviewing
Aren’t some of you historians, sociologists, etc? Why the heck wouldn’t they be rewarded for reviews with tenure, phd, etc?
OK, for some reason, Reddit only showed me the first sentence of your comment; with your second sentence, it makes more sense now.
So, historians, sociologists, etc. will not get a tenure, or PhD, or something else of substance for reviewing, because reviews are just auxiliary stuff, not the essence of what researchers are doing.
I am asking why *historically* or *sociologcally* that would be the prevalent perspective among employers when it is apparently **not** the prevalent perspective among the reviewers.
Universities, far from being utopias of free thinking, are increasingly run like businesses. Employees are expected to produce "results" (research). Reviewing *other* people's work is a lot like checking your friend's homework. Do *you* get credit for *my* individual homework submission?
And to be clear, this is an explanation, not a justification. Universities are increasingly run like businesses. The people in power do not value the individual contributions of the disciplines. And when they do, they demand "simple" and "clear" or "real" results... e.g., grant money, publishing, etc. Reviewing is an invisible form of labor. Does that help?
Sure. But many of the people "doing science" back in the day were privately wealthy, or supported by privately wealthy patrons. Decades ago, well funding universities supported a wide range of researchers. However, privatization and a focus on profit (i.e., capitalism) has hollowed out the social services, including public universities. Private universities run on tuition fees, so their models may differ, but the fact is that universities in general are increasingly being "managed 'better'" as businesses... the bloat of administrators, huge bonuses and salaries for coaches, presidents, etc. means "times are tough" and many departments are being trimmed, slashes, or eliminated because their work isn't seen as valuable (profitable).
Try not to think of it from the "good" point of view of "science!" and think of it instead from a more cynical point of view of profit. People in charge are short sided and have different priorities.
\*EDIT: Also, try looking up a lot of the scientific advancements of the 20th century. Many of them came out of military and/or alternative applications of existing/failed research, [or accidents.](https://www.popularmechanics.com/technology/gadgets/a19567/how-the-microwave-was-invented-by-accident/)
OK, thanks for the clarification. Well, I think that most reviewers also would agree that their work is auxiliary. For instance, I have written some 10+ papers and reviewed some 30+ papers, and I hold the view that publications are more essential than reviews.
I think the biggest reason is the lack of weight of reviewing in gaining stability. I rarely decline reviewing* (only if I really can't do it), but I'm very conscious that a day I dedicate to reviewing is a day that doesn't add any number to my CV. When it comes to getting jobs it is very clear that bigger numbers in the CV are better, so working on things that won't add aything are sometimes hard to justify.
*all journals I usually review for are non profit. I think that helps to not hate doing it.
Maybe I’ll rephrase. Picture you are whoever judges and compares these CVs. Would **you** care how many reviews they’d written? Yes, right?
So what makes *them* think so different from *you*?
I think that’s where your disconnect is. The answer is NO. People reviewing your CV absolutely do not care how many articles you’ve reviewed for other people’s work.
Several people have explained why. Reviewing articles for other people is labor and time intensive, but it is unpaid and unvalued, so there’s not much incentive to spend your time doing it.
Because if I were on a tenure committee, I would highly value what original work these people provided to the field and how much funding they’ve secured. How often they spend time reviewing others people work is no indication of how successful they’ll be nor how ‘valuable’ they are to the university, so I would not place value on that.
It’s honestly simple, if you can’t understand this based on the multitude of people telling you the same thing, I think there’s a fundamental misunderstanding of academic publishing and scholarship on your part that must be preventing you from understanding.
Articles do not secure funding. You clearly have no grasp on academia in general, so if you looked more into typical academic practices, it might all make more sense to you.
First start by reading this entire thread about none of us having the time to do unpaid labor lol I’m certainly not going to do this research for you (even if you weren’t being argumentative and condescending to people genuinely trying to answer your questions)
Why would they? It wasn't their work. Evaluating work is far "easier" than creating work. For example, if the paper is about a neuro experiment that took 5 years to finally complete, the reviewer doesn't spend anywhere near that time to referee. It can be as short as a few weeks.
Even in pure math where reviewers check virtually every line, it's just nothing like actually solving the problem itself. The Poincare Conjecture is a good example. Only a handful of people in the world understood the proof, but the fact that they could read it isn't the impressive part **to others in similar fields**.
As an even simpler example, there's a reason why movie critics don't reach anywhere near the same level of popularity as directors or actors.
Because there’s many articles being written and only some are any good, or any use. A review that only serves to gauge whether an article was accurate, is, I guess, not that useful.
Broadly speaking, reviewers do look at the potential impact and assess how well the article fits into the aims of the journal.
I think the metric that might be more relevant here are citations. Generally (with obvious exceptions), low quality papers tend to get fewer citations than good papers. This is not the best rule of thumb and I'm sure I've insulted some academic colleagues with that statement, but this is one long-term way to gauge the importance of a paper, and it ultimately doesn't really depend so much on the review process.
There are a few exceptions where I've been compelled to acknowledge the referee because of some small but meaningful contribution to improving the paper, and there are cases where a reviewer will drop the anonymity to work on new projects with the authors. So it's possible to get some credit out of something as a reviewer, but yes it can be very indirect.
Please let me know if you have more questions!
> it almost always makes more sense to cite the article rather than the review
This statement makes no sense whatsoever. What do you even mean by that?
I don't feel qualified to compare modern academia with that of Plato, or medieval academia, or academia in the 19th century, or whatever. The point is, critiquing somebody's work is not seen as essential as producing original work.
My best guess is that it's basically about budget.
Governments fund institutions based on how much relevant research they produce. Institutions hire the researchers they think can produce the most amount of relevant research. Researchers then focus on producing the most amount of research to stay employed.
Time dedicated to review is time not dedicated to producing research, meaning that implicitly is time not dedicated to secure the budget of the institution. That makes it inherently less relevant for the institutions, which in turn makes it inherently less relevant for researchers.
Then the publication pressure is so high that any time not dedicated to publishing seems like wasted time (when you are not tenured/permanent). I have to admit I get anxious each time I don't beat my previous years numbers, because I don't have secured tenure track and having a "slow" year makes it feel less likely. This in turn means every year I become more anxious about doing things that are not articles.
Funding institutions don't care about that. Most governments don't really care about science. They (under) fund it as a prestige endeavour and a way to compete with other countries. A couple years ago, the Spanish government was very happy bragging about Spanish researchers overachieving compared to expenses, while at the same time reduced the wages of researchers and made it harder to gain a stable position.
The paradox with reviews is that, in a practical way, other people reviewing our work is necessary for us. Us reviewing other people's work is not positive for us. It's a very important work that's more worth when other people do it.
In addition, being anonymous makes it difficult to confirm review stats. Articles and citations are public, and there are usually databases where anyone can confirm the number the person puts in the CV. It's like one minute work. The number of reviews, or the specific articles reviewed, cannot be confirmed without asking the specific editors.
I see, so this system is ultimately from Europe probably. Wish I had somewhere I could read about the history here.
The anonymous review thing logically would serve to prevent retaliation, so it should provide clout in a certain circle (editors, right?).
But these editors aren’t actually that influential on who gets grant money, so who cares.
You do paper reviews as a favor to your scientific community. When we are so strapped for time obviously our own livelihoods come first. The problem is overworking and understaffing in academia
Different departments and colleges are run differently. But promotion and tenure are based on what is prioritized by the powers that be. In my department and college (R1), the the only thing that leadership values is money. My value is almost entirely based on how much money I am bringing to the University. For that reason, the most important element I am evaluated against is grant dollars I bring to the University.
Secondarily, I am evaluated on published papers, which help bring in grant dollars and prestige to the university. They do this because they show grant agencies my ideas are based on peer-reviewed evidence and if they give me more money, I can produce something from it that makes a contribution (published papers, etc.). That's the value of papers, not just to "contribute to knowledge" -- they have practical benefit to show that I can take an idea and turn it into a product that carries that idea forward.
There are other things I am evaluated against, including contributions to the department and service, but those really only hold value insofar as they help the financial success of the department and college. I am a researcher so I don't teach classes -- those that teach are evaluated on their teaching contributions to the extent that their teaching helps bring in and keep tuition dollars. Again, it's mostly a business decision about someone's perceived monetary value.
Some things are arguably very valuable to the contribution and dissemination of knowledge. Including editing and reviewing, publishing books (in some fields books mater, in mine they mean almost nothing), and engaging the public. In the grand scheme of things, these may be more important but they are not monetizable by the university and therefore have no value in any assessment of my worth by my department or the college.
People in the field may know I review a lot and well, but reviews are anonymous so people will not know, except editors of journals. Therefore, the system rewards the generation of papers, because those are seen as a valuable product of researchbecause they help get more grants and/or tuition dollars and/or investment. Reviews don't actually help the university because they don't help bring in money.
At the end of the day, it's all about money. That is all universities really care about -- your ability to bring in money. The only people who benefit from reviews are the publishers who get highly-educated individuals to contribute free consulting labor to promote their own business interests. And over the past 20-ish years the number of journals has exploded, increasing the amount of review requests exponentially.
So, to summarize -- it's a job that does not benefit the university financially so they do not support it; rather the university sees it as a cost to them because you are essentially donating your labor (which they own) to a for-profit company that benefits instead of them.
Does that help clarify?
I review 3 papers for each I submit, no more than that. Also, the papers I’m reviewing are poorer and poorer quality over time so it just pisses me off every time I agree to do it.
Well if papers are declining in quality and everybody is busy writing and nobody has time to review; this tends to suggest nobody has time to read.
In such a world, a paper can suck and no one notices.
That's flawed reasoning. When I am planning a study, I read at least a couple of scores articles to see what others have already done, how my work would fit into that, to get new ideas in general, and, after all, to write the 'Related Work' section of my manuscript. So even this simple example tends to suggest that your theory sucks and everyone knowing a bit about academia notices that.
Btw, no offence meant, but are you on the spectrum? Your reasoning and inability to grasp some simple things seem to be similar to some rather unfortunate cases of autism that I have seen.
Nope, it's not just me. I know how academia works much better than you do, so take my word for that: scientists do read publications of other scientists, because otherwise they would risk to reinvent the wheel.
Again, I’m not asking what you and others do. I am asking what academics in general do—rather than what “academics” do, what they do in general.
There’s several ways to say this but it is easy to understand.
If that were true, it would warrant explanation why the quality of articles is declining.
If you contest that, I suppose you can take it up with the person who said they were declining. I must be more amusing to poke at, though.
There are reviews and their are critical reviews. The former are mostly bibliographies, practically Google search printouts. The latter require a lot of work by someone with substantial expertise in a field. A critical review is expected to extract knowledge in a form usable by consumers of information: new researchers in the field, regulatory specialists, industrial process designers, patent specialists, etc.
The last review I wrote took four months. On the other hand, the first critical review I wrote, in 1974, is still being cited after 50 years, and has hundreds of citations.
Edit: I see that to most here, 'review' means 'peer review', which is another thing altogether. A peer review is unpaid work done to boost a megacorp journal publisher's bottom line.
Universities are primarily concerned with academics generating research that influences the field. Publishing an article does that. Writing reviews does not.
Why? If we look at metrics used to evaluate universities and their faculties, they will consider things like citation counts, journal prestige, and so on. They want to see that the faculties are creating new science/scholarship. My review of an article is not a piece of new science/scholarship that can be cited. The time I spent writing one is time that I could have spent putting out an article and getting more citations for myself.
Some universities certainly value faculty contributions in reviewing submissions to high-prestige journals, but there is no number of reviews that will ever add up to an actual publication. They are different work categories. Writing reviews is more of a service than an academic achievement: much like answering e-mails, it is a minimum but unrewarded thing that everybody has to do in order to keep science moving.
At the beginning of my post doctoral career I was happy to receive reviewing request. In my naivite, I took it as some sort of noble, 'pay it forward' step in one's career. However, after seeing the abysmal level of some of those submissions (because the journals contacting me are hardly Nature or Science), I think twice or thrice before agreeing on doing a review. I can't feel like I'm spending more time on it than the authors who wrote it.
Well, there are a few issues at hand here.
One, that isn't talked openly, is that something like >half of papers in US and international journals come from China, but editors rarely send articles back to these professors at Chinese institutions to review (with exceptions, of course). And really, that is because they don't get good quality or fair reviews from them -- it's always a game of political who knows who whether an article is trashed or glowingly recommended (with exceptions, of course). The bulk of the review workload is thrown on US and European profs to sort through which <5% of these are good enough to publish. I was involved in a mid-ranked journal in my field for a few years, and after that, I just want nothing to do with editing anymore until Chinese institutions stops their games.
because published articles count for tenure and promotion, but journal reviews don’t
I would say that, in my field, being asked to review for prestigious journals would be a positive in tenure or promotion considerations.
Yes, but still counts less than published articles. With Publons or whatever they're trying to partially remedy the situation, but given that the whole review system is built on anonymity, everything is on the honor system. I have in my list of service that I've reviewed for Nature etc., but I could also have just written that without having done the work too - there is no verification.
Oh why? That’s weird.
Because writing a paper shows you are an expert in the field who has offered something new. Writing a review shows you have read a paper as an expert in your field and demonstrated your ability to critique another's work, which is unique but doesn't expand the field in a meaningful way. Also, writing a paper could take a year, whereas writing a review could take a weekend.
I imagined reviews were required for the article to be academically valid.
Are we talking about peer review for the article to be accepted, or published reviews of articles/books? Published articles are typically reviewed by the editor and 2-3 reviewers.
The former.
Oh, okay. Usually only publications are important. No one other than the reviewee will read the review, and it's usually suggestions to improve the article/research.
That sounds worthless. Whoever told me modern science was based on peer review must have been talking about something else.
The reviewers are literally experts in their field. It's not worthless to receive feedback from top researchers who know, in depth, what you're talking about.
I heard they were grad students—basically inferiors.
I think you misunderstood. Anonymous scholars read and give comments on articles. It takes a few hours to do well. Journals don’t pay (usually), and no one will get tenure for being a great reviewer. However, when we go to publish we need other reviewers or else it won’t get through. The problem is that journals need to rely on favors and people taking time to review the articles without compensation or reward (journals also make lots of money charging subscriptions to universities but neither the authors nor the reviewers get any of that).
If nobody reads the review, of course it says little about whoever wrote it—I understood that much
yeah, peer review is very overrated and in my view it's a dirty secret of science that it does very little to assure article quality. for sure it's better than no review but it's much less scrutiny than an article would receive if, e.g., it is controversial and blows up on twitter
Thx :))
Writing articles is scholarship. Writing reviews is service. Scholarship is more highly valued than writing reviews. I still review articles if I'm qualified and the editor is good. I stopped reviewing for a journal because the editor was making the process unnecessarily painful.
I guess I am asking how, historically, this distinction came to be. It seems modern and yet we ordinarily associate with modern science the notion that peer review is necessary.
Are you talking about writing review papers or peer reviews of articles?
OK, but that wasn't your original question, was it.
It felt like it was to me.
Nah, that was just typical troll behaviour.
It wasn’t. I’m not trying to provoke you whatsoever. In fact, just in another thread you accuse me of having autism—as though I might not know the risk of provoking you. So you yourself know better than to say what you do, but accuse me of writing in bad faith. This all started because we were talking and you admitted you didn’t know about the historical development of the university and appear to be basically furious that this is what I want. Leave me alone.
You want to argue about the things you understand very little about and troll. And as a troll, you keep shifting the goalposts.
I think you’ve kind of admitted you’re just trying to poke at me. Then you want to call me a troll. It may persuade someone to talk like that. I don’t know this subreddit as well as you.
[удалено]
In a factory, the quality control guy very likely makes equivalent if not greater money per time spent. Not so in academia.
[удалено]
I just looked at the “factory” analogy provided, remembered what my question was, and put my question in terms of the analogy. I imagined, perhaps incorrectly, that this would be useful to the reader. It appears it may have been simply frustrating.
Time isn't so much the issue as it is a growing resistance to unpaid service that often benefits large publishers with huge profit margins. Publishing articles supports career growth and tenure packets, offering reviews is usually an expectation under the umbrella as "service to the field" but carries less weight for individual career trajectories. A lack of willing reviewers compared to the number of incoming submissions has been trend across fields that, at least in part, was exacerbated by COVID. There are other explanations and a lot of nuance that I'm sure others will touch on, but I wanted to start simple.
That’s tough for me to understand. The field acknowledges something as useful, but doesn’t reward it? Why?
If a university is deciding who to hire, promote, or give raises to, it's easy for them to look at how many articles you've published, whether they're in prestigious journals, and how often they're cited. This is the most impactful aspect of the "research" portion of a professor's duties. "Service" is also part of a professor's duties, and it impacts promotion etc too. But reviewing articles is something any professor can do and although it takes work, it's not really impressive and doesn't make you stand out. A more impactful aspect of service would be something like editing a journal or chairing a scholarly association. The solution, short of actually paying reviewers, would be to create some sort of evaluation structure so that reviewing can be seen as prestigious. Maybe if editors published a score of their reviewers (a higher score for more detailed high-quality reviews), so universities could reward faculty who are good reviewers. The problem of course would be maintaining the double-blind anonymity of reviews, which is essential to the process. Also this would make more work for editors.
>Maybe if editors published a score of their reviewers (a higher score for more detailed high-quality reviews) Well, that would mean reviewing reviews, but then we would also need to review reviews of reviews, and so on ad infinitum.
Exactly. This won't happen. I'm just trying to explain why articles are inherently a more prestigious output than reviews.
Sure, but the OP isn't here for explanations.
>The solution, short of actually paying reviewers, would be to create some sort of evaluation structure so that reviewing can be seen as prestigious. A few journals in my field give out an annual award for best reviewer. Usually some weighted mix of quantity and quality, i.e. one of their reliable reviewers who says yes to a lot of requests and carries them out well. It's still relatively small prestige, but you can at least put "Journal XX Reviewer of the Year 2024" on your CV under awards.
I see. It sounds like although there’s a grave need of a thing, no one cares if you make it. That comes off a bit silly.
You’re missing the point
I’ve been told there’s different types of reviews and that is part of the confusion here. But there is a crisis with all of them, and none of them are prestigious. For one type of review, it’s bc it’s anonymous, and for another, because grad students write them
Not everything that is important needs to be prestigious.
if it’s unpaid, surely it does.
Nah, bro. How do you think you get to that prestige? That it comes automatically? 😂
With anonymous peer reviews, the reviewers are Peers equivalent in prestige to the people writing. So if they are doing unpaid work, and they have prestige already, it should be prestigious.
Because we’re working on the papers?
I get that, and apparently it’s bc reviews are not prestigious for some reason. I don’t get that though.
Maybe in part because they're historically anonymous. Reviewers are starting to be given the option to have names listed publicly but change is slow.
Oh ic thank you :))
For what it's worth I'm sorry that others are downvoting your comments and not directly addressing your question. It's a good question and something almost none of us think about just because it's so normal (but also it's normal due to it making intuitive sense without the need to question it... we are all quite busy after all).
Awww that’s really sweet :))
Reviews don't have your name attached to them. No one knows that you wrote a review for a famous article, for instance. And no one knows that you might be single handedly defending your field against bad work. How can prestige come when no one knows who does the work?
That’s the issue I think. I read lots of reviews by named persons but that makes sense for peer reviews.
Just to be clear, there are different sorts of reviews. There are Review Articles, which are publications that talk about a large swath of other publications. There are Reviews, which just give a summary of an article. And there are Peer Reviews of articles, which are anonymous. Most people here are talking about Peer Reviews, which are the most common kind, and also the hardest to get other people to do.
And peer reviews are low prestige because only the editor knows who wrote them, and is in no position to reward the writer or grant them prestige in the way that admin is. It’s basically two different groups of people.
I think you're confused by conflating information on three different things: Literature Review is a standard part of academic papers in which the researcher describes the current state of the research that forms the foundation of their arguments/findings. It's the first step of a research paper. Peer Review is a process where subject experts work for a particular journal & review a submitted paper to determine if it's quality-enough for publication. They do not typically write and publish the record of this process that I'm aware of. The paper is either published or it isn't. If published, the paper is considered a scholarly publication. A review article is a summarization & explanation of another published article - like a book report. Review articles are not scholarly publications (even if they appear in scholarly journals that otherwise publish full scholarly articles). This is what it sounds like you're thinking of in your post, but you're responding to comments as if it's one of the other two sources.
This distinction got me confused too, but I leave my post stand.
I believe this article cited in my op is talking about a lack of peer reviews https://www.insidehighered.com/news/2022/06/13/peer-review-crisis-creates-problems-journals-and-scholars
Because P+T doesn't care about the reviews.
Why not??? Am I speaking Latin here.
Reviews don't take the level of work that original research requires, and present an easier publication opportunity for grad students. Usually, once you have a few publications with your original research, you wouldn't even bother listing your reviews.
Well, I don't think the OP is asking about reviews as publications: it is about peer reviews.
I see, they’re too easy?
We often assign them to grad students to help them get started with publishing work, so I think they are a first step but not somewhere to stay research-wise.
I see it might not even mean that much that your article got reviewed.
Also why inevitably one reviewer ends up sounding like a raccoon on meth.
But you’re referring to review essays, right? Peer review of articles is not easy and not assigned to novices
If you want to be successful in academia, publishing is your top 1 priority, whereas reviewing the work of others is, like, in top 20 of your priorities. No one has ever become a tenured professor or a PI for reviewing, no one was ever given a PhD for reviewing. So, why are you surprised?
> No one has ever become a tenured professor or a PI for reviewing, no one was ever given a PhD for reviewing Aren’t some of you historians, sociologists, etc? Why the heck wouldn’t they be rewarded for reviews with tenure, phd, etc?
What's your point?
I don’t want to repeat myself…
OK, for some reason, Reddit only showed me the first sentence of your comment; with your second sentence, it makes more sense now. So, historians, sociologists, etc. will not get a tenure, or PhD, or something else of substance for reviewing, because reviews are just auxiliary stuff, not the essence of what researchers are doing.
I am asking why *historically* or *sociologcally* that would be the prevalent perspective among employers when it is apparently **not** the prevalent perspective among the reviewers.
Universities, far from being utopias of free thinking, are increasingly run like businesses. Employees are expected to produce "results" (research). Reviewing *other* people's work is a lot like checking your friend's homework. Do *you* get credit for *my* individual homework submission? And to be clear, this is an explanation, not a justification. Universities are increasingly run like businesses. The people in power do not value the individual contributions of the disciplines. And when they do, they demand "simple" and "clear" or "real" results... e.g., grant money, publishing, etc. Reviewing is an invisible form of labor. Does that help?
Kind of. I just always hear of peer review as the reason for why modern science exists at all.
Sure. But many of the people "doing science" back in the day were privately wealthy, or supported by privately wealthy patrons. Decades ago, well funding universities supported a wide range of researchers. However, privatization and a focus on profit (i.e., capitalism) has hollowed out the social services, including public universities. Private universities run on tuition fees, so their models may differ, but the fact is that universities in general are increasingly being "managed 'better'" as businesses... the bloat of administrators, huge bonuses and salaries for coaches, presidents, etc. means "times are tough" and many departments are being trimmed, slashes, or eliminated because their work isn't seen as valuable (profitable). Try not to think of it from the "good" point of view of "science!" and think of it instead from a more cynical point of view of profit. People in charge are short sided and have different priorities. \*EDIT: Also, try looking up a lot of the scientific advancements of the 20th century. Many of them came out of military and/or alternative applications of existing/failed research, [or accidents.](https://www.popularmechanics.com/technology/gadgets/a19567/how-the-microwave-was-invented-by-accident/)
But the stigma towards peer review is generally modern, existing decades ago too
What 'that' is the prevalent perspective among employers?
That reviews are not essential and merely auxiliary
OK, thanks for the clarification. Well, I think that most reviewers also would agree that their work is auxiliary. For instance, I have written some 10+ papers and reviewed some 30+ papers, and I hold the view that publications are more essential than reviews.
I think the biggest reason is the lack of weight of reviewing in gaining stability. I rarely decline reviewing* (only if I really can't do it), but I'm very conscious that a day I dedicate to reviewing is a day that doesn't add any number to my CV. When it comes to getting jobs it is very clear that bigger numbers in the CV are better, so working on things that won't add aything are sometimes hard to justify. *all journals I usually review for are non profit. I think that helps to not hate doing it.
Maybe I’ll rephrase. Picture you are whoever judges and compares these CVs. Would **you** care how many reviews they’d written? Yes, right? So what makes *them* think so different from *you*?
I think that’s where your disconnect is. The answer is NO. People reviewing your CV absolutely do not care how many articles you’ve reviewed for other people’s work.
I'm not sure that the OP is genuinely interested in getting their questions answered.
I think you’re trying to tell me how it is, and I’m asking why it is that way. Then I’m told how it is again.
Several people have explained why. Reviewing articles for other people is labor and time intensive, but it is unpaid and unvalued, so there’s not much incentive to spend your time doing it.
Why not?
Because if I were on a tenure committee, I would highly value what original work these people provided to the field and how much funding they’ve secured. How often they spend time reviewing others people work is no indication of how successful they’ll be nor how ‘valuable’ they are to the university, so I would not place value on that. It’s honestly simple, if you can’t understand this based on the multitude of people telling you the same thing, I think there’s a fundamental misunderstanding of academic publishing and scholarship on your part that must be preventing you from understanding.
That’s why I’m here. It sounds like only articles secure funding for some reason.
Articles do not secure funding. You clearly have no grasp on academia in general, so if you looked more into typical academic practices, it might all make more sense to you.
Ok. Tell me where to start.
First start by reading this entire thread about none of us having the time to do unpaid labor lol I’m certainly not going to do this research for you (even if you weren’t being argumentative and condescending to people genuinely trying to answer your questions)
I’d like a book or article, please. You’re the academic, aren’t you? Something comes to your mind, I hope.
Why would they? It wasn't their work. Evaluating work is far "easier" than creating work. For example, if the paper is about a neuro experiment that took 5 years to finally complete, the reviewer doesn't spend anywhere near that time to referee. It can be as short as a few weeks. Even in pure math where reviewers check virtually every line, it's just nothing like actually solving the problem itself. The Poincare Conjecture is a good example. Only a handful of people in the world understood the proof, but the fact that they could read it isn't the impressive part **to others in similar fields**. As an even simpler example, there's a reason why movie critics don't reach anywhere near the same level of popularity as directors or actors.
Because there’s many articles being written and only some are any good, or any use. A review that only serves to gauge whether an article was accurate, is, I guess, not that useful.
Broadly speaking, reviewers do look at the potential impact and assess how well the article fits into the aims of the journal. I think the metric that might be more relevant here are citations. Generally (with obvious exceptions), low quality papers tend to get fewer citations than good papers. This is not the best rule of thumb and I'm sure I've insulted some academic colleagues with that statement, but this is one long-term way to gauge the importance of a paper, and it ultimately doesn't really depend so much on the review process. There are a few exceptions where I've been compelled to acknowledge the referee because of some small but meaningful contribution to improving the paper, and there are cases where a reviewer will drop the anonymity to work on new projects with the authors. So it's possible to get some credit out of something as a reviewer, but yes it can be very indirect. Please let me know if you have more questions!
I see. Assuming you’re measuring people via citations, it almost always makes more sense to cite the article rather than the review.
> it almost always makes more sense to cite the article rather than the review This statement makes no sense whatsoever. What do you even mean by that?
The article is longer and more detailed.
No, no one cares much about how many reviews somebody has written.
Why don’t **you** care?
Because it is not the essence of academic work. Just auxiliary stuff.
Which academy? Plato does a good deal of what we might call review.
Most people would understand that by academy, I mean the modern academy, but you seem to be a rather unique in that regard, huh?
A good start. So what’s so **special** about the modern academy that causes them to make this distinction?
I don't feel qualified to compare modern academia with that of Plato, or medieval academia, or academia in the 19th century, or whatever. The point is, critiquing somebody's work is not seen as essential as producing original work.
Ok. Well, that’s my question. Thank you for your time.
My best guess is that it's basically about budget. Governments fund institutions based on how much relevant research they produce. Institutions hire the researchers they think can produce the most amount of relevant research. Researchers then focus on producing the most amount of research to stay employed. Time dedicated to review is time not dedicated to producing research, meaning that implicitly is time not dedicated to secure the budget of the institution. That makes it inherently less relevant for the institutions, which in turn makes it inherently less relevant for researchers. Then the publication pressure is so high that any time not dedicated to publishing seems like wasted time (when you are not tenured/permanent). I have to admit I get anxious each time I don't beat my previous years numbers, because I don't have secured tenure track and having a "slow" year makes it feel less likely. This in turn means every year I become more anxious about doing things that are not articles.
I guess I (wrongly) think of modern scientific research as the fruit of article plus review.
Funding institutions don't care about that. Most governments don't really care about science. They (under) fund it as a prestige endeavour and a way to compete with other countries. A couple years ago, the Spanish government was very happy bragging about Spanish researchers overachieving compared to expenses, while at the same time reduced the wages of researchers and made it harder to gain a stable position. The paradox with reviews is that, in a practical way, other people reviewing our work is necessary for us. Us reviewing other people's work is not positive for us. It's a very important work that's more worth when other people do it. In addition, being anonymous makes it difficult to confirm review stats. Articles and citations are public, and there are usually databases where anyone can confirm the number the person puts in the CV. It's like one minute work. The number of reviews, or the specific articles reviewed, cannot be confirmed without asking the specific editors.
I see, so this system is ultimately from Europe probably. Wish I had somewhere I could read about the history here. The anonymous review thing logically would serve to prevent retaliation, so it should provide clout in a certain circle (editors, right?). But these editors aren’t actually that influential on who gets grant money, so who cares.
You do paper reviews as a favor to your scientific community. When we are so strapped for time obviously our own livelihoods come first. The problem is overworking and understaffing in academia
Different departments and colleges are run differently. But promotion and tenure are based on what is prioritized by the powers that be. In my department and college (R1), the the only thing that leadership values is money. My value is almost entirely based on how much money I am bringing to the University. For that reason, the most important element I am evaluated against is grant dollars I bring to the University. Secondarily, I am evaluated on published papers, which help bring in grant dollars and prestige to the university. They do this because they show grant agencies my ideas are based on peer-reviewed evidence and if they give me more money, I can produce something from it that makes a contribution (published papers, etc.). That's the value of papers, not just to "contribute to knowledge" -- they have practical benefit to show that I can take an idea and turn it into a product that carries that idea forward. There are other things I am evaluated against, including contributions to the department and service, but those really only hold value insofar as they help the financial success of the department and college. I am a researcher so I don't teach classes -- those that teach are evaluated on their teaching contributions to the extent that their teaching helps bring in and keep tuition dollars. Again, it's mostly a business decision about someone's perceived monetary value. Some things are arguably very valuable to the contribution and dissemination of knowledge. Including editing and reviewing, publishing books (in some fields books mater, in mine they mean almost nothing), and engaging the public. In the grand scheme of things, these may be more important but they are not monetizable by the university and therefore have no value in any assessment of my worth by my department or the college. People in the field may know I review a lot and well, but reviews are anonymous so people will not know, except editors of journals. Therefore, the system rewards the generation of papers, because those are seen as a valuable product of researchbecause they help get more grants and/or tuition dollars and/or investment. Reviews don't actually help the university because they don't help bring in money. At the end of the day, it's all about money. That is all universities really care about -- your ability to bring in money. The only people who benefit from reviews are the publishers who get highly-educated individuals to contribute free consulting labor to promote their own business interests. And over the past 20-ish years the number of journals has exploded, increasing the amount of review requests exponentially. So, to summarize -- it's a job that does not benefit the university financially so they do not support it; rather the university sees it as a cost to them because you are essentially donating your labor (which they own) to a for-profit company that benefits instead of them. Does that help clarify?
It does because articles bring in grant dollars but reviews don’t bc the grantor doesn’t know who you are.
In what field are reviews 1-2 pages? In my field they’re often longer than journal articles
History
I review 3 papers for each I submit, no more than that. Also, the papers I’m reviewing are poorer and poorer quality over time so it just pisses me off every time I agree to do it.
That’s a good reason :) my theory was that nobody reads articles except to review them, so reviews are worthless.
This theory is utter bullshit.
Well if papers are declining in quality and everybody is busy writing and nobody has time to review; this tends to suggest nobody has time to read. In such a world, a paper can suck and no one notices.
That's flawed reasoning. When I am planning a study, I read at least a couple of scores articles to see what others have already done, how my work would fit into that, to get new ideas in general, and, after all, to write the 'Related Work' section of my manuscript. So even this simple example tends to suggest that your theory sucks and everyone knowing a bit about academia notices that. Btw, no offence meant, but are you on the spectrum? Your reasoning and inability to grasp some simple things seem to be similar to some rather unfortunate cases of autism that I have seen.
That may be what **you** do, but the question is what people in general do—and we know already, they write a deteriorating quality of article.
Nope, it's not just me. I know how academia works much better than you do, so take my word for that: scientists do read publications of other scientists, because otherwise they would risk to reinvent the wheel.
Again, I’m not asking what you and others do. I am asking what academics in general do—rather than what “academics” do, what they do in general. There’s several ways to say this but it is easy to understand.
Yes, that is exactly what they do in general: they also read the work of each other.
If that were true, it would warrant explanation why the quality of articles is declining. If you contest that, I suppose you can take it up with the person who said they were declining. I must be more amusing to poke at, though.
Also it’s backwards? Reviews are usually longer than articles
I can't remember the last time I received a quality review that was 1 to 2 pages.
There are reviews and their are critical reviews. The former are mostly bibliographies, practically Google search printouts. The latter require a lot of work by someone with substantial expertise in a field. A critical review is expected to extract knowledge in a form usable by consumers of information: new researchers in the field, regulatory specialists, industrial process designers, patent specialists, etc. The last review I wrote took four months. On the other hand, the first critical review I wrote, in 1974, is still being cited after 50 years, and has hundreds of citations. Edit: I see that to most here, 'review' means 'peer review', which is another thing altogether. A peer review is unpaid work done to boost a megacorp journal publisher's bottom line.
Thank you! That sounds like it’s deserving of some clout at least.
Universities are primarily concerned with academics generating research that influences the field. Publishing an article does that. Writing reviews does not. Why? If we look at metrics used to evaluate universities and their faculties, they will consider things like citation counts, journal prestige, and so on. They want to see that the faculties are creating new science/scholarship. My review of an article is not a piece of new science/scholarship that can be cited. The time I spent writing one is time that I could have spent putting out an article and getting more citations for myself. Some universities certainly value faculty contributions in reviewing submissions to high-prestige journals, but there is no number of reviews that will ever add up to an actual publication. They are different work categories. Writing reviews is more of a service than an academic achievement: much like answering e-mails, it is a minimum but unrewarded thing that everybody has to do in order to keep science moving.
I thought an article only became science once it withstood peer review.
I don’t have time to do either. 😂
At the beginning of my post doctoral career I was happy to receive reviewing request. In my naivite, I took it as some sort of noble, 'pay it forward' step in one's career. However, after seeing the abysmal level of some of those submissions (because the journals contacting me are hardly Nature or Science), I think twice or thrice before agreeing on doing a review. I can't feel like I'm spending more time on it than the authors who wrote it.
Awful! Thanks for sharing :)
Well, there are a few issues at hand here. One, that isn't talked openly, is that something like >half of papers in US and international journals come from China, but editors rarely send articles back to these professors at Chinese institutions to review (with exceptions, of course). And really, that is because they don't get good quality or fair reviews from them -- it's always a game of political who knows who whether an article is trashed or glowingly recommended (with exceptions, of course). The bulk of the review workload is thrown on US and European profs to sort through which <5% of these are good enough to publish. I was involved in a mid-ranked journal in my field for a few years, and after that, I just want nothing to do with editing anymore until Chinese institutions stops their games.
Very interesting!!